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SUMMARY 


This  technical  paper  documents  the  development  and  use  of  mission  effectiveness  (ME)  charts 
as  the  primary  organizational  feedback  tool  in  the  Methodology  for  Generating  Efficiency  and 
Effectiveness  Measures  (MGEEM)  system.  ME  charts  are  central  to  the  development  of  a 
comprehensive  management  information  system  with  features  which  make  possible  improved 
leadership,  enhanced  motivation,  and  continual  Improvement  of  work  processes. 

The  purpose  of  this  paper  is  to  provide  the  MGEEM  facilitator  with  a  guide  for  developing  and 
using  ME  charts.  Through  the  MGEEM  process,  the  commander  or  manager  of  the  target  organization, 
his  inmediate  subordinates,  and  customers  define  the  organizational  key  result  areas  (KRAs).  A 
second  group  consisting  of  subordinates  and  workers  develops  performance  indicators  for  each  KRA 
and  then  develops  for  each  indicator  an  ME  chart  which  relates  levels  of  performance  to  mission 
effecti  veness. 

In  explaining  the  use  of  ME  charts,  this  paper  also  provides  procedures  for  combining  the 
performance  data  for  two  or  more  organizational  units  (e.g.,  two  branches  within  a  division)  into 
a  single  measure  of  performance.  It  also  describes  how  to  construct  Management  Engineering 
Program  Feedback  Charts  for  use  in  tracking  an  organization's  performance  over  time. 

Several  ME  chart  exercises  and  suggested  solutions  are  Included  as  facilitator  aids  to 
implementing  an  effective  MGEEM  system. 


PREFACE 


The  work  documented  in  this  technical  paper  supports  the  transition  of  an 
AFHRL-developed  organizational  performance  measurement  and  enhancement  technology.  The 
transition  office  is  the  Air  Force  Management  Engineering  Agency,  with  AFHRL  providing 
continuing  research  support.  Effective  development  and  aggregating  of  measures  of 
organizational  performance  will  enable  the  Air  Force  and  other  000  agencies  to  carry  out 
their  mission  responsibilities  in  an  effective  manner. 

The  authors  express  their  appreciation  to  individuals  who  participated  In  the 
review  of  this  document.  In  particular,  we  thank  Maj  Donald  M.  Riemensnider,  Chief  of 
the  Program  Management  Division,  and  CMSgt  Joseph  F.  Dymon,  Chief  of  the  Research  and 
Programs  Integration  Branch,  both  at  Headquarters,  Air  Force  Management  Engineering 
Agency  (AFMEA).  Others  who  provided  significant  Inputs  to  the  final  manuscript  are  Cdr 
Frederick  C.  Orton,  Aircraft  Maintenance  Training  and  Readiness  Officer,  Staff  of  the 
Commander,  Anti-Submarine  Warfare  Wing,  US  Pacific  Fleet;  Ms  Teresa  M.  Fazio, 
Productivity  Director,  Naval  Air  Test  Center;  and  Ms  Sandra  Edsall,  Productivity 
Monitor,  Naval  °lant  Representati ve  Office,  McDonnell -Douglas  Corporation;  and  TSgt 
Russell  D.  Chauncey,  Management  Engineering  Technician,  Air  Force  Security  Police 
Management  Engineering  Team  'AFSPMET).  In  addition,  this  effort  benefited  from 
conversations  with  Dr.  William  E.  Alley  of  the  AFHRL  Manpower  and  Personnel  Division. 
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METHODOLOGY  FOR  GENERATING  EFFICIENCY  AND 
EFFECTIVENESS  MEASURES  (MGEEM):  A  GUIDE  FOR  THE 
DEVELOPMENT  AND  AGGREGATION  OF  MISSION  EFFECTIVENESS  CHARTS 


I.  INTRODUCTION 

The  Methodology  for  Generating  Efficiency  and  Effectiveness  Measures  (MGEEM)  is  a  set  of 
technologies  which  makes  possible  the  development  in  any  organization  of  a  conplete  management 
information  system.  A  trained  measurement  facilitator  uses  the  MGEEM  to  guide  target 
organization  members  to  identify  the  organization's  principal  Intended  accom>l ishments,  called 
key  result  areas  (KRAs),  and  measures  of  organizational  performance  of  the  KRAs,  called 
performance  indicators.  The  facilitator  also  leads  the  organization  to  develop  ways  of  using 
indicator  performance  results  to  provide  feedback  to  managers  to  improve  tneir  leadership  and  to 
workers  to  stimulate  their  motivation,  and  to  serve  as  a  basis  for  continually  improving  work 
processes.  Two  forms  of  feedback  (the  objectives  matrix  and  the  line  graph)  were  identified  in 
Tuttle  and  Weaver  (1986).  The  present  paper  Introduces  mission  effectiveness  (ME)  charts,  an 
improved  procedure  for  providing  indicator  performance  feedback.  It  also  provides  a  practical 
guide  to  measurement  facilitators  in  the  use  of  these  charts.  (Note:  An  earlier  version  of  the 
ME  chart  was  called  a  contingency  chart.) 

In  implementing  MGEEM,  a  measurement  facilitator  may  use  several  alternative  techniques  for 
providing  feedback.  The  sinplest  of  these  is  the  line  graph;  the  more  complex  are  the  objective 
matrix  (discussed  in  Tuttle  and  Weaver,  1986)  and  the  ME  chart.  When  deciding  which  of  these 
three  techniques  to  use,  the  facilitator  should  consider  that  target  organizations  differ  as  to 
the  sophistication  of  their  members  with  respect  to  measurement. 

For  military  organizations,  it  Is  usually  possible  to  develop  a  complete  MGEEM  system  of 

KRAs,  Indicators,  and  ME  charts;  but  some  organizations  whose  members  have  less  measurement 
sophistication  mav  reaulre  an  interim  orocess  consisting  only  of  KRAs,  indicators,  and  simple 
line  graphs  for  feedback.  Later,  after  sufficient  learning  has  taken  place  among  organization 
members,  the  facilitator  can  substitute  the  more  sophisticated  and  useful  format  for  feedback. 

The  facilitator  should  also  consider  that  the  feedback  alternatives  vary  with  respect  to 
their  ease  of  use,  their  use  of  weighting  to  distinguish  among  the  importance  of  the  indicators, 
their  ability  to  deal  with  the  complex  relationships  between  indicator  performance  and  overall 
organization  effectiveness,  their  capacity  for  addressing  the  interactions  among  indicators,  and 
the  comprehensibility  of  their  results  to  members  of  the  target  organization.  Each  alternative 
has  strong  points,  and  the  facilitator  is  encouraged  to  become  familiar  with  the  relative 

strengths  of  each  so  that  the  most  appropriate  technique  may  be  selected  for  the  implementation 
at  hand. 

The  following  discussion  assumes  that  the  reader  is  familiar  with  Tuttle  and  Weaver  (1986, 

Section  4.2)  and  presents  ME  charts  as  an  alternative  way  to  report  performance  results. 


II.  DEVELOPING  MISSION  EFFECTIVENESS  CHARTS 

Mission  effectiveness  chart  development,  like  KRA  and  indicator  development,  requires  for  its 
accomplishment  consensus  among  members  of  the  target  organization.  In  terms  of  the  Measurement 
Development  Teams  (MDTs)  suggested  by  Tuttle  and  Weaver  (1986,  Section  3.2),  Team  B  is  the  one 
responsible  for  developing  these  charts.  (Note:  Team  A  consists  of  upper-level  management  and 
immediate  subordinates.  Team  B  consists  of  Immediate  subordinates  and  key  workers.) 


Refining  Indicators 


To  facilitate  the  ME  chart  development  process,  indicators  should  be  refined  when  they  are 
first  identified  and  defined.  This  adds  a  refinement  step  to  the  Indicator  Development  process 
described  in  Section  3.4  of  Tuttle  and  Weaver  (1986).  It  is  better  to  refine  indicators  at  this 
initial  stage  of  indicator  development  rather  than  later  because  Team  B  will  have  recently 
completed  identifying  and  defining  Indicators  and,  consequently,  will  have  a  clear  understanding 
of  their  meaning.  If  this  refinement  is  deferred,  some  members  of  the  team  may  forget  important 
details  about  the  indicators. 

After  indicator  development  has  gone  through  the  final  step  of  documentation,  as  suggested  by 
Tuttle  and  Weaver  (1986,  p.  28),  the  facilitator  should  lead  Team  B  to  consensus  on  the  answers 
to  three  questions  for  each  indicator.  During  this  facilitation,  an  individual  should  act  as  a 
recorder  to  document  the  information  as  it  Is  decided  upon.  (This  individual  should  not  be  the 
facilitator  or  a  member  of  Team  B.)  The  first  two  questions  to  be  used  to  refine  each  indicator 
are  as  follows: 

1.  IF  EVERYTHING  GOES  RIGHT  FOR  YOUR  ORGANIZATION,  WHAT  IS  THE  "FEASIBLE  BEST"  YOU 

COULD  DO  ON  HIS  INDICATOR? 

2.  IF  EVERYTHING  GOES  WRONG  FOR  YOUR  ORGANIZATION,  WHAT  IS  THE  "FEASIBLE  WORST"  YOU 
COULD  DO  ON  THIS  INDICATOR? 

This  pair  of  values  constitutes  the  best  and  worst  performance  possible  considering  real-world 
organizational  constraints  and  environmental  peculiarities. 

The  facilitator  should  ensure  that  Team  B  understands  that  “feasible"  best  and  worst  are  not 

the  same  as  "absolute"  best  and  worst.  For  Instance,  consider  this  indicator  for  customer 

satisfaction  taken  from  Tuttle  and  Weaver  (1986,  Table  11,  p.  28):  "number  of  customer  inquiries 

satisfied/number  of  customer  Inquiries  received  (monthly)  x  100."  The  absolute  or  arithmetic 
best  would  be  1002,  meaning  that  all  inquiries  received  are  satisfied.  This  may,  however,  not  be 
the  feasible  best  if  Team  B  believes  that  the.,  regularly  receive  a  few  inquiries  which  cannot  be 
satisfied  regardless  of  how  hard  they  try.  The  feasible  best  may,  thus,  be  only  952  or  even  90S. 

The  third  question  to  be  asked  is: 

3.  BETWEEN  THE  FEASIBLE  BEST  AND  FEASIBLE  WORST,  WHAT  IS  THE  LEVEL,  POINT,  OR  ZONE  OF 
PERFORMANCE  ON  THIS  INDICATOR  THAT  IS  ,vuT  GOOD  OR  NOT  BAO,  THAT  IS  THE  "BREAK-EVEN" 

OR  "DON’T  ROCK  THE  BOAT"  POINT? 

The  facilitator  should  explain  to  Team  3  that  this  is  also  the  safety,  expected,  or  indifference 
point.  That  Is,  this  is  the  point  at  which  the  work  center  supervisor  will  neither  bec'Hne 
concerned  that  performance  Is  too  low  and  commit  more  resources,  nor  feel  that  performance  is  so 
good  that  the  work  center  should  be  recognized  for  outstanding  effort. 

For  instance.  In  terms  of  the  customer  satisfaction  indicator  example  above.  Team  B  may  feel 
that  satisfying  8  of  10  customer  inquiries  (80S)  would  be  neither  bad  nor  good.  That  is,  the 
team  may  believe  that  the  supervisor  will  become  concerned  if  performance  falls  below  80S,  but 
that  if  performance  rises  above  80S,  the  supervisor  will  begin  to  recognize  that  the  work  center 
is  doing  a  good  job.  On  the  other  hand.  Team  8  may  believe  that  there  is  no  single  indifference 
point,  but  Instead,  there  is  a  range  of  Indifference  represented  by  a  zone  of,  say,  from  75S  to 
85S.  In  other  words,  they  believe  that  the  supervisor  is  largely  indifferent  in  a  zone  of  75S  to 
85S  but  will  become  concerned  if  the  percentage  falls  below  75S  or  be  Impressed  if  the  percentage 


rises  above  85$,.  The  system  can  readily  accommodate  a  zone  of  indifference  in  lieu  of  a  point  if 
the  real -life  conditions  are  best  depicted  in  this  manner. 


Weighting  Indicators 

Once  KRAs  and  performance  indicators  have  been  agreed  upon  and  all  Indicators  refined  for 
feasible  best/worst  and  indifference  points,  the  next  step  Is  for  the  facilitator  to  point  out  to 
Team  B  the  likelihood  that  their  indicators  are  not  of  equal  Importance.  The  facilitator  then 
supports  this  assertion  with  several  simple  examples.  For  instance,  in  assessing  the  overall 
performance  of  a  baseball  player,  which  of  these  indicators  would  be  of  greater  interest  to  the 
team  manager:  “number  of  home  runs  hit”  or  "number  of  bats  broken"?  In  a  military 
communications-navigatlon  equipment  repair  shop,  would  the  supervisor  be  more  Interested  in 
"radios  and  radars  returned  "rom  the  flight  line  for  failure  to  work  properly  (bounce  rate)"  or 
"nunber  of  dental  checkup  appointments  missed  by  personnel"?  In  answering  this  second  question, 
remember  that  although  missing  dental  appointments  is  serious,  the  bounce  rate  directly  affects 
flying  capability  and  is  therefore  of  critical  Interest  to  the  wing  commander,  whose  success 
depends  in  large  part  on  getting  aircraft  into  the  air  during  operational  readiness  exercises. 
(Note:  The  answers  are,  of  course,  that  the  baseball  manager  is  more  interested  in  home  runs  and 
the  supervisor  is  more  concerned  about  the  bounce  rate.) 

Getting  Team  B  to  achieve  consensus  on  the  relative  importance  of  their  indicators  is  the 

next  step  in  the  development  of  ME  charts.  The  process  of  making  judgments  about  the  relative 

importance  of  indicators  Is  expedited  by  use  of  an  Indicator  Weighting  Table,  which  the 

facilitator  presents  to  Team  8  on  a  chalkboard.  (See  Table  1.) 

Table  1.  Example  of  an  Indicator  Weighting  Table 

Feasible  Effectiveness 

Worst/Best  Ranks  Points 

Worst  Best  Worst  Best  Worst  Best 

_ 0)  (2)  (3)  (4)  (S)  (6) 

KRA  f] .  Customer  Satisfaction 

Indicator  *1.  No.  of  customer  50  95  5  3  -SO  75 

inquiries  satisfied/no.  of 
customer  inquiries  received  x  100. 

Indicator  #2.  No.  of  complaints  5043  -70  75 

recei  ved. 

KRA  »2.  Timely  Completion  of  Taskings 

Indicator  #3.  Tasking  completed  60  90  3  3  -75  75 

on  time/total  tasking  x  100. 

KRA  #3.  Ensure  Compliance  with 
AFR  175-37. 


Indicator  #4.  Exercise  ratings 

75 

100 

1 

1 

-90 

100 

passed/total  ratings  x  100. 

Indicator  #5.  No.  of  severe 

80 

0 

2 

2 

-80 

90 

discrepancies  found /no.  of 
discrepancies  found  x  100. 


On  the  left-hand  side  of  the  Indicator  Weighting  Table,  the  facilitator  writes  the  KRAs  and 
indicators  which  were  developed  for  the  target  organization  by  Teams  A  and  B,  respectively. 
These  KRAs  and  the  indicators  which  measure  KRA  accomplishment  constitute  the  rows  of  the  table. 
At  the  top  of  the  table,  the  facilitator  then  enters  the  column  headings  shown  in  Table  1.  For 

the  first  and  second  columns,  the  facilitator  writes,  for  each  indicator,  the  feasible  worst  and 

feasible  best  values  derived  earlier  in  the  process  of  refining  indicators.  The  facilitator 
should  remind  Team  B  that  these  feasible  worst  and  best  values  are  their  own  judgments  and  are 
different  from  the  absolute  (arithmetic)  worst  and  best. 

In  the  example  shown  in  Table  1,  there  are  3  KRAs  and  5  indicators;  for  each  indicator  there 
is  a  corresponding  set  of  feasible  worst  and  best  values  shown  in  columns  (1)  and  (2).  For 
Indicator  #1,  Team  B  said  in  the  indicator  refinement  stage  that  the  feasible  best  the  work 

center  could  do  in  satisfying  inquiries  was  95%;  they  believed  that  some  peculiarity  would  always 
prevent  their  achieving  100%.  They  said  the  feasible  worst  was  50%.  On  Indicator  #2,  the 

feasible  best  and  worst  in  terms  of  number  of  conplaints  received  were  0  and  5,  respectively;  and 
so  on  for  the  feasible  best  and  worst  for  the  other  three  indicators. 

Once  the  facilitator  has  entered  all  feasible  worst  and  best  values  values  in  columns  (1)  and 
(2),  the  next  step  is  to  complete  column  (4)  of  the  Indicator  Weighting  Table.  This  column  is 
used  to  record  the  consensus  judgment  of  Team  B  as  to  the  relative  importance  or  rank  of  each 
feasible  best  value  of  the  indicators.  To  secure' judgments  about  the  ranks  of  the  feasible  best 
values  for  column  (4),  the  facilitator  says; 

SUPPOSE  EVERYTHING  IMAGINABLE  GOES  RIGHT  FOR  THE  WORK  CENTER  FOR  A  GIVEN  MEASUREMENT 
PERIOD.  YOU  ARE  STAFFED  AT  100  PERCENT.  LITTLE  SICK  Ok  "iri'JAL  LEAVE  IS  TAKEN. 

THERE  ARE  NO  READINESS  EXERCISES.  OTHER  ORGANIZATIONS  YOU  WORK  WITH  ARE 

COOPERATIVE.  BECAUSE  OF  THESE  ANO  OTHER  POSITIVE  INFLUENCES,  THE  WORK'  CENTER 
PERFORMS  AT  ITS  FEASIBLE  BEST  ON  ALL  INDICATORS.  THE  WORK  CENTER  PERFORMS  AT  95%  ON 
CUSTOMER  INQUIRIES  SATISFIED,  0  ON  CUSTOMER  COMPLAINTS,  AND  SO  ON.  IF  THIS  WERE 
TRUE,  WHICH  OF  THE  5  FEASIBLE  BESTS  WOULD  HAVE  THE  GREATEST  POSITIVE  EFFECT  ON  THE 
OVERALL  PERFORMANCE  (MISSION)  OF  THE  WORK  CENTER? 

Group  B  will  then  discuss  alternative  answers  to  this  question  until  the  members  reach  consensus 
as  to  wnich  indicator  has  the  most  important  feasible  best.  This  indicator  is  ranked  1  and  a  “1" 
is  entered  in  column  (4)  for  that  indicator.  (In  the  Table  I  example,  the  most  important 

feasible  best  was  that  for  Indicator  *4.)  In  case  of  ties  between  two  or  more  indicators,  each 
of  tne  tied  indicators  will  oe  assigned  the  same  rank.  For  instance,  if  two  indicators  will 

eoually  result  in  the  greatest  positive  impact,  both  should  receive  the  rank  of  1.  The 
facilitator  continues: 

NOW  THAT  YOU  HAVE  IDENTIFIED  THE  INDICATOR  WHOSE  FEASIBLE  BEST  HAS  THE  GREATEST 
POSITIVE  IMPACT,  WHICH  INDICATOR'S  FEASIBLE  BEST  HAS  THE  SECOND  GREATEST  POSITIVE 
IMPACT  ON  THE  OVERALL  PERFORMANCE  OF  THE  WORK  CENTER? 

This  indicator  Is  ranked  2  and  a  "2“  is  marked  in  the  appropriate  row  of  column  (4).  (In  Table 
1,  Indicator  45  is  ranked  2.)  The  process  continues,  with  Team  B  ranking  the  importance  of  the 
feasible  best  on  the  remaining  indicators. 

After  the  feasible  bests  have  been  ranked  in  column  (4),  the  facilitator  asks  Team  B  to 

change  these  ranks  to  effectiveness  points  for  column  (6),  and  says: 


IF  W£  AUTOMATICALLY  ASSIGN  TOO  EFFECTIVENESS  POINTS  TO  THE  FEASIBLE  BEST  FOR  THE 

INDICATOR  YOU  RANKED  FIRST,  AS  HAVING  THE  GREATEST  POSITIVE  MISSION  IMPACT,  HOW  MUCH 
LESS  IS  THE  IMPACT  ON  THE  OVERALL  MISSION  OF  THE  INDICATOR  WHOSE  FEASIBLE  BEST  YOU 
RANKED  SECOND? 

The  facilitator  then  explains  that  If  the  indicator  ranked  second  were  only  about  half  as 
important  to  the  mission  as  the  indicator  ranked  first,  it  would  be  assigned  50  effectiveness 
points.  If  it  were  considered  three-fourths  as  important,  it  would  receive  75  points.  If  its 
impact  were  almost  as  important  as  that  of  the  indicator  ranked  first,  an  effectiveness  rating  of 
95  or  98  points  might  be  assigned.  This  process  continues  until  all  indicator  ranks  in  column 
(4)  have  been  transformed  to  effectiveness  points  and  recorded  In  column  (6). 

For  the  indicators  shown  In  Table  1,  Indicator  #4  (with  the  rank  of  1)  automatically  received 

100  effecti veness  points.  Indicator  #5  (with  the  rank  of  2)  was  judged  to  have  90?  as  positive 
an  impact  on  the  work  center's  effectiveness  as  Indicator  #4  and  was  thus  assigned  90 

effectiveness  points.  Finally,  Indicators  #1,  #2,  and  #3  were  tied  in  column  (4)  at  the  rank  of 
3,  and  Team  B  agreed  that  these  indicators  should  receive  75  effectiveness  points  each. 

A  similar  process  is  then  performed  to  determine  the  ranks  (column  3)  and  effectiveness 
points  (column  5)  of  the  feasible  worsts.  The  facilitator  begins  this  process  of  ranking  with 
Team  B  as  follows: 

SUPPOSE  EVERYTHING  GOES  WRONG  FOR  THE  WORK  CENTER  FOR  A  GIVEN  MEASUREMENT  PERIOD. 

YOUR  MANNING  LEVEL  IS  VERY  LOW.  THERE  IS  A  BIG  SNOW  STORM.  THERE  IS  AN  UNEXPECTED 
READINESS  EXERCISE.  THE  MOON  IS  FULL.  MORA' E  IS  LOW  AND  THE  WORK  CENTER  OPERATES 
AT  ITS  FEASIBLE  WORST  ON  ALL  5  INDICATORS.  IF  THIS  WERE  TRUE,  WHICH  OF  THE  FEASIBLE 
WORSTS  WOULD  HAVE  THE  GREATEST  NEGATIVE  IMPACT  ON  THE  OVERALL  PERFORMANCE  (MISSION) 

OF  THE  WORK  CENTER?  WHICH  LOW  SCORE  WOULD  HURT  YOU  THE  MOST? 

The  feasible  worsts  are  ranked  in  column  (3)  with  "1"  for  the  one  with  the  greatest  negative 

impact;  again,  ties  are  possible.  As  shown  in  column  (3)  of  Table  1,  for  feasible  worsts  the 

team  judged  that  Indicator  #4  would  have  the  greatest  negative  impact;  Indicator  #5,  the  second 
greatest  negative  impact;  Indicator  #3,  the  third;  Indicator  #2,  the  fourth;  and  Indicator  *1, 
the  fifth. 

The  next  step  is  to  transform  the  ranks  in  column  (3)  to  oegati ve  effectiveness  points  for 

column  (5).  Though  the  feasible  worst  will  usually  -eceive  -100  effectiveness  points,  the 

automatic  assignment  of  -100  is  not  absolutely  necessary  to  transform  to  effectiveness  points  the 
indicator  with  the  feasible  worst  rank.  For  instance,  in  the  case  of  Indicator  »4  in  Table  1 
(which  was  ranked  1  for  feasible  worst).  Team  B  believed  that  the  feasible  worst  was  simply  not 
as  bad  as  -100.  They  saw  a  difference  In  the  Impact  on  the  work  center's  effectiveness  (mission) 
between  the  feasible  best  and  worst  on  this  Indicator.  They  believed  that  -90  was  the  Impact  of 
the  feasible  worst;  therefore,  -90  was  recorded  in  column  (5)  for  Indicator  f4. 

Once  the  negative  effectiveness  rating  Is  determined  for  the  feasible  worst  ranked  number  1, 
the  process  continues,  with  Team  3  coming  to  consensus  regarding  the  ratings  for  the  remaining 
indicators.  In  the  Table  1  example,  Team  B  decided  that  the  next  feasible  worst  indicator  (#5) 
should  receive  -80  effectiveness  points  and  that  the  other  three  indicators  (#3,  f2,  and  #1 ) 
should  be  assigned  effectiveness  points  of  -75,  -70,  and  -50,  respectively. 

It  should  be  noted  that  the  effectiveness  points  for  the  feasible  bests  are  always  positive 
(e.g.,  100,  90,  and  75),  and  the  effectiveness  points  for  the  feasible  worsts  are  always  negative 
(e.g.,  -90,  -80,  -75,  -70,  and  -50).  In  the  unlikely  event  that  a  work  center  has  only  one 


indicator,  its  feasible  best  is  assigned  100  effectiveness  points  and  its  feasible  worst  is 
assigned  the  appropriate  negative  effectiveness  points  relative  to  -100. 

This  completes  the  discussion  of  weighting  indicators  using  an  Indicator  Weighting  Table. 
The  role  of  feasible  bests,  feasible  worsts,  indifference  points  (or  zones),  and  effectiveness 
points  (weights)  in  constructing  ME  charts  will  now  be  explained. 

Plotting  Known  Values  for  Mission  Effectiveness  Charts 

After  refining  the  indicators  and  assigning  relative  weights  to  each,  ar  suggested  in  the  two 
previous  sections  of  this  paper,  the  facilitator  proceeds  to  the  last  step  in  the  development  of 
an  ME  chart  for  each  indicator;  namely,  assisting  Group  B  to  specify  the  slopes  for  each  chart. 
To  do  so,  the  facilitator  begins  by  showing  them  how  to  plot  their  previously  determined  values 
on  the  chart. 

The  general  form  of  an  ME  chart  is  shown  in  Appendix  A.  The  vertical  axis  of  an  ME  chart 
shows  an  indicator's  overall  effecti veness  or  mission  impact  and  is  scaled  from  -100  through  0  to 
+100  in  increments  of  10.  The  vertical  axis  is  the  same  for  all  ME  charts.  The  horizontal  axis 
is  used  to  record  those  values  specific  to  a  gi /en  indicator;  these  values  will  range  from  the 
feasible  worst  to  the  feasible  best  for  that  particular  indicator.  It  is  recommended  that  the 
facilitator  prepare  a  transparency  of  the  chart  contained  in  Appendix  A  for  use  on  an  overhead 
projector  to  assist  Team  B-  in  plotting  these  values. 

Using  the  vugraph  of  the  chart  from  Appendix  A  as  an  example,  the  facilitator  tells  Team  B 
that  they  are  at  last  ready  to  construct  an  ME  chart  for  their  Indicator  #1.  The  facilitator 
further  explains  that  this  chart  will  serve  as  a  vehicle  for  providing  feedback  to  them  as  to 
their  performance  on  this  indicator  and  as  to  how  their  level  of  performance  impacts  the  overall 
performance  (mission)  of  the  work  center  according  to  the  values  shown  on  the  vertical  axis. 

Team  8  members  are  then  shown  that  they  made  some  important  decisions  about  the  chart  wnen 
they  defined  the  feasible  best  and  worst  for  the  indicator,  in  that  these  values  represent  the 
"endpoints"  of  its  horizonta'  axis.  Thereupon,  the  facilitator  labels  the  horizonal  axis  with 
the  name  of  the  indicator  in  question,  marks  the  feasible  best  as  the  next-to-the-highest  point 
(to  the  right)  on  the  horizontal  axis,  and  the  feasible  worst  as  the  next-to-the-1  owest  point  (to 
the  left)  on  the  horizontal  axis.  (The  feasible  best  and  worst  could  go  in  the  extreme  highest 
and  lowest  points  on  the  horizontal  axis,  but  leaving  a  space  at  either  end  of  the  axis  makes  a 
more  easily  readable  presentation.) 

Next,  the  facilitator  fills  in  the  intervals  along  the  horizontal  axis  between  the  feasible 
worst  and  the  feasible  best.  When  under  pressure  to  keep  the  process  going,  the  facilitator  may 
find  it  difficult  to  precisely  scale  between  the  two  extremes,  but  Team  B  members  will  be 
understanding  and  satisfied  with  a  serie*  of  approximations.  Later,  when  the  ME  charts  are 
automated  or  typed,  an  exact  horizontal  scale  can  be  developed  and  the  entire  ME  chart  can  be 
presented  for  Team  B's  review.  Figure  1  shows  such  a  scaling  of  the  horizontal  axis  of  Indicator 
#1  from  Table  1. 

Next,  the  facilitator  explains  to  Team  B  that  they  have  already  developed  three  points  for 
the  ME  chart  that  represent  three  levels  of  performance  on  the  indicator.  These  points  are  the 
indifference  point  (or  zone)  and  effectiveness  values  for  the  feasible  worst  and  best  discussed 
previously. 

Figure  2  depicts  these  three  parts  of  the  curve  for  the  Table  1  example.  For  this  particular 
example,  fn  which  Team  9  decided  earlier  that  their  supervisor  would  be  indifferent  if  ’5%  to  95* 


of  customer  inquiries  were  satisfied,  the  zone  from  75  to  85  would  represent  the  indifference 
zone,  where  performance  has  no  impact  (i.e.,  0  effectiveness  points)  on  the  work  center's  overall 


M  100  | 

I  90  | 

S  80  | 

S  70  j 

I  60  | 

0  50  | 

N  40  | 

30  | 

E  20  | 

F  10  | 

F  o  | - 

E  -10  | 

C  -20  | 

T  -30  | 

I  -40  j 

V  -50  | 

E  -60  | 

N  -70  | 

£  -80  j. 

S  -90  | 

S  -1 00  | _ 

50  60  70  80  90  95 

No.  of  Customer  Inquiries  Satisfied 

-  x  100 

No.  of  Customer  Inquiries  Received 

Figure  1.  Customer  Satisfaction  Mission  Effectiveness  Chart  with 
Horizontal  Axis  Scaled. 


effectiveness  (mission).  This  zone  is  shown  by  the  dark  line  in  Figure  2.  Further,  Team  B  in 
that  example  must  be  reminded  that  they  decided  the  comparative  contribution  to  overall  work 
center  effectiveness  (mission)  of  the  feasible  best  (95%)  should  be  assigned  75  effectiveness 
points  anc  the  feasible  worst  (50%)  should  be  assigned  -50  effectiveness  points.  These  points 
(-50  and  75)  and  the  indifference,  zone  (0  points  each  for  75,  80,  and  85)  for  this  example  are 
plotted  in  Figure  2. 


Specifying  Slopes  for  Management  Effectiveness  Charts 

As  mentioned,  the  last  step  Involved  in  constructing  the  Initial  ME  chart  for  an  indicator  is 
to  specify  the  slope  of  the  ME  curve.  This  Involves  determining  the  intermediate  points  between 
these  three  plotted  values;  that  is,  determining  the  negative  impact  values  between  the  feasible 
worst  and  the  Indifference  point  (zone)  and  the  positive  Intact  values  between  the  indifference 
point  (zone)  and  the  feasible  best.  Though  an  ME  curve  may  be  a  straight  line  between  the  three 
previously  plotted  values,  this  Is  rarely  the  case.  Rather,  for  each  value  shown  on  the 
horizontal  axis  a  separate  judgment  must  be  made  as  to  how  much  that  particular  level  of 
performance  would  negatively  or  positively  affect  overall  mission  performance. 


M 

I 

S 

S 

I 

0 

N 

E 

F 

F 

E 

C 

T 

I 

V 

E 

H 

E 

S 

S 


100 

90 

80 

70 

60 

50 

40 

30 

20 

10 

0 

-10 
-20 
-30 
-40 
-50 
-60 
-70 
-80 
-90 
-1 00 


No.  of  Customer  Inquiries  Satisfied 
————————————————  X  1 00 

No.  of  Customer  Inquiries  Received 

Figure  2.  Indifference  Zone  and  Feasible  Worst  and  Best 
Data  Points  Plotted  for  Customer  Satisfaction. 


Before  asking  Team  B  to  plot  the  intermediate  points  for  its  first  chart,  the  facilitator 
should  provide  and  interpret  several  alternative  ME  charts  to  Illustrate  the  concept.  Figure  3 
shows  two  such  alternatives  (a  dashed-line  curve  and  a  solid-line  curve),  as  applied  to  the  three 
plotted  values  presented  in  Figure  2. 

The  dashed-line  curve  suggests  that  a  very  broad  range  of  performance  on  the  indicator  (from 
552  to  942  inquiries  satisfied)  has  no  really  positive  or  negative  impact  on  mission,  in  that  the 
effectiveness  points  vary  only  from  -10  to  10.  Performance  above  or  below  this  range,  however, 
has  a  substantial  Impact  on  the  mission  (75  and  -50  effectiveness  points,  respectively). 

The  solid-line  curve  In  Figure  3  exhibits  a  very  different  pattern.  According  to  this  curve. 
Immediately  below  the  Indifference  zone  (less  than  752  Inquiries  satisfied),  the  mission  impact 
becomes  severe,  falling  from  0  to  almost  -50  effectiveness  points  for  a  mere  difference  of  52  in 
Indicator  performance.  Below  the  702  level,  however,  the  Impact  is  already  so  bad  that  It  can 
get  only  slightly  worse.  On  the  positive  side,  when  performance  exceeds  the  Indifference  zone 
(greater  than  852),  the  positive  Impact  on  the  mission  is  dramatic;  that  is,  an  increase  from  852 
to  902  Inquiries  satisfied  produces  an  increase  in  effectiveness  points  from  0  to  almost  75. 
Above  the  902  level,  the  Increase  in  mission  impact  is  positive  but  small  by  comparison. 
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Figure  3.  Alternative  Slopes  of  Mission  Effectiveness  Charts 
for  Customer  Satisfaction. 

As  part  of  the  Instruction  concerning  alternatives,  the  facilitator  should  stress  to  Team  B 
that  the  slope  of  the  curve  Is  a  graphic  expression  of  their  policy  about  the  relationship 
between  performance  on  the  indicator  and  the  impact  on  mission  of  various  levels  of  performance. 

Team  B  members  learn  to  quickly  come  to  consensus  about  the  shape  of  curves  for  ME  charts, 
but  the  facilitator  may  wish  to  begin  development  of  the  first  chart  by  asking  questions  such  as 
these: 

IF  THE  IMPACT  ON  OVERALL  EFFECTIVENESS  OF  75%  TO  85%  INQUIRIES  SATISFIED  IS  ZERO 
EFFECTIVENESS  POINTS,  HOW  MUCH  BETTER  IS  IT  TO  HAVE  90%  OF  INQUIRIES  SATISFIED?  HOW 
MUCH  WORSE  IS  IT  TO  HAVE  ONLY  70%  SATISFIED?  65%  SATISFIED?  60%  SATISFIED? 

Answers  to  each  question  form  data  points  which,  when  connected,  result  in  the  construction  of 
the  ME  curve. 

After  the  ME  chart  for  the  first  indicator  is  completed,  the  facilitator  should  then  guide 
Team  B  in  a  similar  manner  through  completion  of  the  ME  charts  for  the  remaining  indicators. 

Checking  Mission  Effectiveness  Charts 

Before  the  MGEEM  system  is  submitted  to  management  for  review,  members  of  Team  B  should 
conduct  two  "sanity  checks"  of  their  ME  charts  to  ensure  that  the  charts  are  both  accurate  and 
logical. 


o 


The  first  check  should  come  immediately  after  the  charts  are  put  into  the  form  in  which  they 
will  be  used  for  providing  feedback  to  workers  and  management.  (This  form  can  range  from  a  fully 
automated  procedure  on  a  computer  to  simple  typed  copies  which  are  photocopied  for  each 
measurement  period.)  At  this  time,  Team  B  members  should  convene  to  review  their  charts  in  order 
to  ensure  that  all  elements  have  been  recorded  and  reproduced  as  originally  specified. 

Another  such  meeting  to  correct  and  modify  charts  should  take  place  when  actual  performance 
results  on  indicators  are  available  from  the  first  measurement  period.  At  this  stage.  Team  B 
members  should  feel  free  to  modify  their  ME  charts  in  any  way  necessary. 

In  addition  to  improving  the  quality  of  charts,  these  meetings  offer  the  added  benefit  of 
getting  organization  members  into  the  habit  of  meeting  to  discuss  their  charts.  Once  the  system 
is  in  full  operation,  monthly  meetings  to  review  performance  on  ME  charts  are  essential  and  are 
the  most  powerful  feature  of  MGEEM.  Feedback  is  widely  believed  to  be  the  most  Important 
motivation  to  the  enhancement  of  performance.  Monthly  feedback  meetings  are  also  used  to 
identify  constraints  to  performance  and  to  discuss  "how  to  work  smarter." 

Facilitator  Responsibilities  in  Securing  Management  Approval 

After  Team  B  is  satisfied  with  its  ME  charts,  the  complete  system  of  KRAs,  Indicators,  and 
charts  is  briefed  to  the  appropriate  higher  level  of  management.  Because  MGEEM  maintains  and 
supports  the  prerogatives  of  traditional  military  management,  the  facilitator  should  ensure  that 
some  cautions  are  observed  in  this  approval  process. 

First,  the  facilitator  should  pre-brief  the  commander  or  manager  to  reaffirm  the  basic 
philosophy  of  MGEEM.  A  key  point  in  MGEEM  implementation  Is  that  the  greatest  gain  in 
organizational  performance  comes  from  increasing  worker  motivation  and  restoring  or  Increasing 
workers'  pride  in  their  work  and  sense  of  craftsmanship.  These  factors  are  especially  important 
in  a  resource-constrained  environment.  In  any  case,  management  should  be  reminded  that  workers 
must  feel  a  sense  of  ownership  of  the  MGEEM  system  if  its  feedback  component  is  to  be 
motivational.  Workers  "buy  into  the  MGEEM  system"  by  creating  the  system  themselves.  Thus, 
management  must  appreciate  that  the  central  Issue  is  not  whether  the  system  measures  with  great 
precision  but  that  it  provides  a  platform  or  vehicle  for  continual  improvement.  Although  overall 
accuracy  is  important,  fine-tuning  oy  management  should  be  resisted. 

Continual  improvement  is  the  essence  of  good  management.  So,  it  really  does  not  matter  if  a 
feasible  best  is  65  or  70  or  that  an  indifference  point  is  25  or  30.  What  important,  however, 
is  that  workers  accept  feedback  from  the  ME  charts  and  are  willing  to  use  them  to  identify  better 
ways  of  doing  business.  And  it  is  appropriate  and  reasonable  for  management  to  disagree  with  and 
ask  questions  about  the  system  so  long  as  workers  maintain  a  feeling  of  ownership;  in  fact, 
management  Interest  is  stimulating  to  workers. 

Certainly,  management  must  be  assured  that  the  system  is  measuring  the  Important  components 
of  the  mission;  for  if  the  system  does  not  capture  the  important  components  of  the  mission,  then 
time  and  resources  could  be  Improperly  diverted  by  the  system  into  less  important  areas  of  work. 
However,  if  management  changes  the  system  unnecessarily— perhaps  merely  to  show  who  is 
"boss"— workers  will  lose  their  sense  of  ownership  and  become  resigned  to  "business  as  usual," 
where  they  do  only  what  they  are  told  and  without  much  enthusiasm.  They  will  be  unwilling  to 
make  suggestions  about  improvement.  This,  of  course,  would  mean  that  the  main  purpose  of  the 
MGEEM  project  has  been  defeated.  Another  way  for  MGEEM  to  be  defeated  is  for  management  to  be 
unresponsive  to  worker  suggestions  about  how  to  improve  the  processes  by  which  work  is 
conducted.  Rather  than  being  rule-bound  and  rigid,  managers  must  play  a.  supportive  role  and  be 
willing  to  endorse  improvement. 


A  second  concern  of  the  facilitator  should  be  to  convince  the  commander  and  other 
higher-level  managers  not  to  expect  to  routinely  monitor  work  center  ME  charts.  ME  charts  are 
not  part  of  any  reporting  requirement  to  higher  management.  Instead,  management  will  receive 
aggregated  or  “rolled-up"  results  of  performance  for  use  as  a  management  tool,  as  will  be 
explained  later.  Managers  therefore  need  to  appreciate  that  the  charts  are  intended  for  use  only 
by  the  work  center  supervisor  and  workers  as  tools  to  improve  work  center  performance. 

The  facilitator  should  also  make  it  clear  to  personnel  Involved  in  an  M6EEM  implementation 
that  they  need  not  fear  management  disapproval  of  the  MGEEM  system  submitted  for  review.  In 
fact,  such  disapproval,  if  handled  correctly,  can  prove  to  be  one  of  the  most  valuable  facets  of 
an  MGEEM  implementation.  Initial  disapproval  requires  that  managers  and  workers  engage  in  a 

dialogue  to  seek  agreement  on  the  details  of  the  system.  After  reflection,  most  workers  welcome 
this  dialogue  because  management  is  expressing  an  Interest  in  their  work  and  they,  in  turn,  have 
an  opportunity  to  present  their  viewpoint  to  management.  Managers  also  welcome  such  dialogue 
because  its  underlying  purpose  is  Improving  organizational  performance,  which  is,  of  course,  of 
vital  importance  to  them.  In  such  discussions,  both  sides  usually  make  concessions  to  some 
degree  and  consensus  is  soon  reached  on  the  final  system  to  be  used. 

As  mentioned  earlier,  MGEEM  supports  the  prerogatives  of  traditional  military  management. 

That  is,  the  commander  maintains  the  right  to  approve  or  modify  the  MGEEM  system.  Consequently, 
the  facilitator  should  present  the  MGEEM  to  workers  realistically  and  in  this  light.  The 
facilitator  should  stress  that  MGEEM  represents  an  opportunity  for  workers  to  make  suggestions  to 
management  about  the  best  way  to  measure  and  improve  organizational  performance.  The  facilitator 
should  create  a  positive  outlook,  with  the  expectation  that  most  suggestions  will  likely  be 

accepted  by  management,  especially  since  the  manager  is  a  member  of  the  team  (Team  A)  that 
initially  developed  the  KRAs.  Although  workers  are  encouraged  to  develop  a  sense  of  ownership  of 
the  system,  the  facilitator  must  also  ensure  they  are  fully  aware  of  management  prerogatives  and 
do  not  naively  believe  that  everything  they  suggest  will  automatically  be  accepted  by 

management.  If  the  facilitator  allows  workers  to  believe  that  acceptance  of  their  suggestions 
will  be  automatic,  changes  which  result  from  management  review  are  likely  to  surprise  and 
disillusion  workers.  Then,  instead  of  having  workers  who  are  interested  and  motivated,  the 
opposite  may  occur. 


III.  AGGREGATING  MISSION  EFFECTIVENESS  CHARTS 

Among  the  most  appealing  features  of  MGEEM  is  that  it  provides  the  capability  to  combine  ME 
effectiveness  scores  from  different  ME  charts.  The  system  makes  it  possible  to  determine  overall 
performance  by  adding  measures  on  several  different  indicators,  such  as  percent  of  suspense  dates 
met  and  number  of  action  items  accomplished.  Further,  through  aggregation,  the  system  also  makes 
it  possible  to  measure  the  combined  performance  of  several  unlike  work  centers,  such  as  avionics 
and  supply  squadrons.  There  are  a  number  of  different  organizational  settings  within  which 
aggregation  is  useful.  One  such  setting  is  the  work  center  itself. 


Aggregation  Within  a  Work  Center 


Aggregation  would  be  useful  to  the  work  center  manager  who  wishes  to  have  a  single  measure  or 
index  of  the  overall  work  center  performance.  Consider,  for  example,  a  work  center  with  2  KRAs 
and  3  indicators,  whose  ME  charts  (one  for  each  Indicator)  are  shown  in  Figure  4.  Suppose  for 
the  current  measurement  period  the  performance  of  this  work  center  is  100  effectiveness  Doints  on 
Indicator  *1,  Percent  Time  Spent;  100  on  Indicator  *2,  Percent  Suspense  Dates  Met;  and  94  on 
Indicator  *3,  'lumber  of  Action  Items  Completed.  The  work  center  manager  would  like  to  know  how 
well  the  work  center  is  doing  overall. 
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figure  x.  Aggregating  Within  a  Wort  Center. 
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The  Aggregation  Problem 


How  is  it  possible  to  aggregate  the  three  levels  of  performance  in  Figure  4  (100,  100,  ana 
94)  into  a  single  number  that  constitutes  a  measure  of  the  overall  productivity  of  the  work 
center?  The  three  measures  cannot  simply  be  added  together  because  they  are  on  different 
scales.  The  result  of  simply  adding  them  together  would  be  uninterpretable. 

The  Mission  Effectiveness  Chart  Solution 


The  answer  to  the  aggregation  problem  would  be  to  use  ME  charts  to  convert  indicator  values 
(on  different  horizontal  scales)  to  a  scale  they  share  in  common  (effectiveness  points  on  the 
vertical  axis).  Consider  the  example  in  Figure  4.  For  each  performance  indicator  value  (100, 
100,  and  94)  on  the  horizontal  axis  of  each  respective  chart,  there  is  a  corresponding  value  in 
effectiveness  points  on  the  vertical  axis.  On  Indicator  #1,  the  100$  in  time  spent  translates  to 
75  effectiveness  points.  On  Indicator  #2,  the  100$  on  suspense  dates  met  constitutes  80 
effectiveness  points.  And,  on  Indicator  #3,  the  94  action  items  completed  reads  as  -40 
effectiveness  points.  These  three  effectiveness  values  (75,  80,  and  -40)  can  be  arithmetically 
summed  to  provide  a  current  effectiveness  score  (CES),  a  value  of  115,  which  represents  the 
overall  performance  of  the  work  center  for  the  current  measurement  period.  Aggregation  was  made 
nocsihle  because  each  of  the  values  (75,  80,  and  -40)  was  changed  to  a  common  metric. 

This  aggregation  procedure  has  important  advantages.  First,  it  takes  into  account  the  fact 
that  indicators  may  not  be  of  equal  importance.  In  the  example.  Indicator  #1  is  the  most 
important  since  its  feasible  best  and  worst  would  result  in  100  and  -100  effectiveness  points, 
respectively.  Second,  it  allows  indicators  to  be  measured  on  any  scale  since  ME  charts  convert 
them  to  a  common  scale.  Third,  the  procedure  is  unaffected  by  the  existence  of  non- 
straight-line  relationships  between  the  Indicators  and  mission  effectiveness  ME,  as  is  the  case 
for  Indicators  #2  and  #3  in  Figure  4. 

The  aggregation  of  effectiveness  points  for  the  ME  charts  within  a  work  center  provides  the 
CES  score  that,  in  itself,  is  very  useful  in  assessing  work  center  performance.  Such  scores  are 
especially  useful  when  compared  at  different  points  in  time  (e.g.,  monthly).  Aggregation  may  oe 
even  more  meaningful  to  the  manager  if  applied  in  a  slightly  different  manner,  as  explained  below. 


The  Management  Engineering  Program  (MEP)  Feedback  Chart 

A  means  of  graphically  displaying  CES  scores  is  the  Management  Engineering  Program  (MEP)' 
Feedback  Chart^ .  Use  of  the  MEP  procedure  involves  defining  a  baseline,  as  shown  in  the 
example  MEP  Feedback  Chart  in  Figure  5.  In  Figure  5,  which  is  based  on  the  information  depicted 
in  Figure  4,  the  baseline  CES  of  115  is  simply  defined  as  100  and  plotted  for  July,  the 
measurement  period  involved.  There  is  no  formula  involved  in  this  step.  The  baseline  CES  is 
always  defined  as  100.  Thus,  in  this  case,  115  =  100.  Each  succeeding  month's  CES  is  converted 
by  a  simple  calculation  to  percentages  relative  to  the  baseline  of  115  =  100.  Any  successive 
month's  CES  can  be  changed  to  a  percentage  for  use  in  the  feedback  chart  by  dividing  the  CES  for 
the  current  period  by  the  CES  for  the  baseline  period  and  multiplying  by  100,  as  shown  in  the 
following  equation: 


Current  Period  CES 


Baseline  CES 


X  100  =  MEP  Value 


^The  MEP  Feedback  Chart  was  suggested  by  Maj  Don  M.  Riemensnider,  Chief  of  the  Program 
Management  Division  of  the  Air  Force  Management  Engineering  Agency  (AFMEA),  and  bears  the  name  of 
the  program  .vhere  the  procedure  is  used,  the  Management  Engineering  Program  :MEP). 
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Figure  5.  Example  HEP  Feedback  Chart. 

For  Figure  5,  suppose  that  In  August  the  CES  Increases  from  115  to  130.  The  current  period 
CES  of  130  Is  made  comparable  to  the  baseline  CES  by  130/115  X  100  *  113.  The  value  113  Is  then 
plotted  for  August  on  the  MEP  Feedback  Chart  In  Figure  5.  Similarly,  a  CES  of  150  for  the  month 
of  September  would  be  made  comparable  to  the  baseline  CES  of  115  by  150/115  X  100  =  130.  The 
value  130  Is  then  plotted  for  September  on  the  chart.  For  each  successive  period,  the  current 
period  changes  but  the  MEP  Feedback  Chart's  baseline  number  in  the  denominator  remains  the  same. 
That  is,  all  values  to  be  plotted  on  the  chart  are  computed  relative  to  the  baseline. 

For  the  MEP  Feedback  Chart  shown  in  Figure  5,  the  baseline  was  determined  based  on  the  first 

month's  CES.  .However,  In  order  to  have  a  more  representative  baseline,  data  for  several  months 

may  also  be  used  in  establishing  the  baseline  value.  For  example,  if  the  MGEEM  system  is 

developed  using  data  that  already  exist  In  the  information  system  of  the  target  organization,  it 
may  be  possible  to  reconstruct  a  baseline  period  CES  value  by  computing  the  average  of  the 

monthly  CESs  for  a  period  of  months  prior  to  the  month  of  the  MGEEM  implementation.  If 
sufficient  historical  data  do  not  exist,  another  option  is  also  possible.  After  the  MGEEM  has 
been  in  operation  for  a  period  of  time,  a  baseline  can  be  redefined  as  the  average  of  CESs  for, 
say,  the  first  6  months  or  a  year.  However,  it  Is  well  to  remember  that  the  baseline  should 
represent  a  typical  or  average  period  of  organizational  performance.  If  the  baseline  is  set  In  a 
period  that  is  atypical  of  performance,  subsequent  percentages  for  CESs  will  be  unrealistically 
above  or  below  the  baseline.  If  it  is  necessary  that  the  baseline  be  established  during  such  a 
period  (either  above  or  below  average),  this  fact  must  be  considered  in  making  interpretations. 


Aggregation  Across  Work  Centers 

Aggregation  would  also  be  useful  to  managers  of  two  or  more  work  centers  who  wish  to  have  a 
single  <ndex  of  the  combined  overall  performance  of  these  organizations.  This  measure  might  be 
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used  for  a  branch,  division,  or  higher  organizational  unit.  For  the  sake  of  illustration, 
consider  Figure  6,  which  shows  Branch  A  composed  of  two  work  centers,  one  with  3  ME  charts  and 
one  with  5  ME  charts. 


Branch  A 


Work  Center  1 


100 


Indicator  #1 


Work  Center  2 


100 


Indicator  #1 


7^ 
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80 


Indicator  #2 
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80 


50 


Indicator  #5 


Figure  6.  Aggregating  Across  Work  Centers. 

The  obvious  answer  to  developing  one  number  as  an  index  of  overall  performance  would  appear 
to  be  to  simply  aggregate  the  effectiveness  scores  from  the  8  ME  charts  as  was  explained  In  the 
previous  section  dealing  with  aggregation  within  one  work  center,  but  things  are  not  as  simple  as 
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they  seem.  If  we  suppose  the  CESs  (effectiveness  scores  for  current  performance)  in  the 
illustration  in  Figure  6  for  a  given  month  are  75,  80,  and  -40  for  the  ME  charts  of  Work  Center  1 
and  90,  50,  50,  -10,  22  for  the  ME  charts  of  Work  Center  2,  why  not  simply  add  these  CESs  for 
both  work  centers  together  for  a  total  of  317  and  post  this  total  on  a  MEP  Feedback  Chart?  To  do 
so  would  be  a  serious  error  in  that  their  relative  contributions  to  Branch  A's  productivity  have 
not  been  taken  into  account. 


The  Unequal  Contribution  Problem 


It  is  an  error  to  aggregate  across  work  centers  by  simply  adding  CESs,  such  as  those  for  the 
two  work  centers  In  Figure  6.  We  can  combine  the  scores  for  Work  Center  1  or  the  scores  for  Work 
Center  2,  but  we  cannot  combine  the  scores  for  the  two  work  centers  as  these  work  centers  do  not 
contribute  equally  to  branch  productivity.  Consider  that  if  both  of  these  work  centers  did  their 
feasible  bests,  the  maximum  effectiveness  score  (MES)  for  Work  Center  1  would  be  200  MES  (100  + 
80  +  20),  whereas  the  MES  for  Work  Center  2  would  be  370  MES  (100  +  90  +  80  +  50  +  50).  The 
Branch  Chief  may  regard  both  work  centers  as  equally  important  in  their  contributions  to  the 
branch  mission,  or  the  Branch  Chief  may  consider  the  contribution  of  Work  Center  1  to  be  twice  as 
important  as  the  contribution  of  Work  Center  2.  In  either  case,  it  is  inaccurate  to  allow  Work 
Center  2  to  contribute  a  potential  maximum  of  370  points  while  the  equally  important  or  twice  as 
important  Work  Center  1  contributes  a  potential  maximum  of  only  200  effectiveness  points.  This 
is  the  problem  of  "unequal  contribution,"  and  it  arises  when  the  MESs  of  organizations  to  be 
aggregated  are  not  in  proportion  to  the  management's  assessment  of  their  relative  contributions 
to  the  mission  of  the  organization  within  which  they  are  to  be  aggregated. 


The  Aggregation  Correction  Factor  (ACF) 

A  solution  to  the  problem  of  unequal  contribution  has  been  found  and  should  be  used  to 
compare  organizational  units  after  all  the  ME  charts  have  been  developed  for  each  of  the 
organizational  units  to  be  aggregated.  It  should  be  noted  that  the  aggregation  correction  factor 
(ACF)  Is  used  only  for  purposes  of  aggregating  across  different  organizational  units.  It  is 
important  to  stress  that  the  procedures  used  in  developing  ME  charts,  and  the  use  of  these  charts 
within  a  single  work  center,  are  not  affected  by  the  computations  needed  to  aggregate  across  work 
centers. 

In  order  to  apply  the  ACF,  the  facilitator  must  meet  with  the  manager  of  the  organizational 
units  to  be  aggregated  and  must  obtain  a  judgment  about  the  relative  contribution  of  each 
organizational  unit  to  the  mission  of  the  next  hierarchical  level.  For  the  example  Illustrated 
in  Figure  6,  the  facilitator  would  meet  with  the  Branch  Chief  to  ask  which  of  the  two  work 
centers  contributes  most  to  the  accomplishment  of  the  branch  mission.  The  two  work  centers  could 
contribute  equally  or  unequally. 

If  the  Branch  Chief  believes  they  contribute  unequally,  he/she  is  asked  to  assign  ranks:  1 
for  the  most  important  work  center,  2  for  the  second  most  Important,  and  so  on.  Next,  the  Branch 
Chief  is  asked  to  assign  percentages  to  the  ranks,  with  rank  1  automatically  set  to  100%.  In  the 

example  in  Figure  6,  if  Work  Center  1  is  ranked  1  and  Work  Center  2  ranked  2,  then  Work  Center  1 

would  be  set  to  100%.  If  the  8ranch  Chief  believes  Work  Center  2  is  only  half  as  important  as 
Work  Center  1,  Work  Center  2  would  be  assigned  a  value  of  50%.  If  the  work  centers  are  deemed  of 
equal  Importance,  each  would  be  assigned  100%.  Or,  if  there  were  three  work  centers  and  one 

contributed  twice  as  much  as  the  other  two,  the  most  Important  would  be  set  at  100%,  with  the 

others  tied  at  50%. 

To  further  Illustrate  the  ACF,  assume  the  Branch  Chief  believes  Work  Center  2  is  only  half  as 
important  to  the  branch  mission  as  Work  Center  1.  Therefore,  since  the  MES  for  the  feasible  best 
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of  Work  Center  1  is  200  (100  +  80  +  20),  the  MES  for  the  feasible  best  for  Work  Center  2  should 
be  only  half  of  200,  or  100.  To  compute  the  ACF  for  reducing  the  370  MES  for  Work  Center  2  to 
half  of  200,  or  100  MES,  the  following  formula  would  be  used: 

MES  Before  Adjustment 

- - -  =  Aggregation  Correction  Factor  (ACF) 

MES  After  Adjustment 

For  Work  Center  2,  the  MES  before  adjustment  is  370  (100  +  90  +  80  +  50)  and  the  MES  after 
adjustment  is  100.  Thus,  370/100  =  3.7  is  the  ACF. 

The  second  step  in  aggregating  contributions  across  these  work  centers  is  to  apply  the  ACF  to 
the  actual  performance  scores  for  Work  Center  2.  To  do  this,  the  202  CES  (90,  50,  50,  -10,  and 
22)  for  Work  Center  2  is  divided  by  the  ACF  (3.7)  to  compensate  for  the  fact  that  the 
contributions  of  Work  Center  2  are  only  half  as  Important  as  those  of  Work  Center  1.  The 
adjusted  CES  for  Work  Center  2  (54.6)  can  then  be  added  to  the  115  CES  (75,  80,  and  -40)  for  Work 
Center  1  to  arrive  at  the  branch  CES  of  169.6  (54.6  +  115). 

For  another  example  of  using  the  ACF  to  solve  the  unequal  contribution  problem,  assume  the 
Branch  Chief  considers  that  the  two  work  centers  in  Figure  6  make  equal  contributions  to  the 
branch  mission.  In  such  a  case,  the  370  MES  of  Work  Center  2  needs  modification  to  equal  the  200 
MES  of  Work  Center  1.  The  MESs  for  both  work  centers  should  be  made  equal  if  they  are  of  equal 
importance  to  the  branch;  thus,  both  work  centers  should  have  MESs  of  200.  Next,  remember  that 
the  CES  for  Work  Center  1  is  115  and  for  Work  Center  2  is  202.  To  make  the  two  work  centers 
contribute  eaually,  the  202  CES  for  Work  Center  2  needs  to  be  adjusted  with  the  ACF.  The 
computation  of  the  ACF  in  this  case  is  370/200  *  1.85.  This  ACF  is  then  used  to  adjust  the  202 

CES  (202/1.85)  and  results  in  109.  The  109  adjusted  CES  for  Work  Center  2  can  now  be  added  to 

the  115  CES  from  Work  Center  1  to  get  224  (109  +  115),  the  CES  for  the  branch. 

(Note:  If  the  Branch  Chief  believes  the  two  work  centers  are  equally  Important,  the  MES  for 

the  feasible  best  of  both  work  centers  should  be  equal.  Work  Center  1  * s  200  MES  could  be 

adjusted  upward  to  equal  the  370  MES  of  Work  Center  2,  or  the  370  MES  for  Work  Center  2  could  be 
adjusted  downward  to  equal  the  200  MES  of  Work  Center  1.  For  the  sake  of  simplicity,  we  chose  to 
present  the  downward  adjustment  only.) 


The  MEP  Feedback  Chart 


Once  the  problem  of  unequal  contribution  has  been  corrected,  it  is  possible  to  use  the  MEP 
Feedback  Chart  format  described  earlier.  To  demonstrate  this,  consider  the  example  above  where 
the  contributions  of  both  work  centers  are  of  equal  importance  and  the  branch  CES  is  224.  To  use 
the  MEP  Feedback  Chart,  the  CES  of  224  is  defined  as  the  baseline  (i.e.,  100).  Afterward,  if  the 
branch  CES  increases  from  224  to  240,  the  branch  point  on  the  feedback  chart  would  Increase  from 
the  baseline  of  100  to  107  (240/224  X  100).  Then,  if  during  the  next  measurement  period  the 
branch  CES  increased  from  240  to  250,  the  new  monthly  value  for  the  feedback  chart  would  be  112 
(250/224  X  100).  In  other  words,  each  increase  (or  decrease)  is  computed  relative  to  the 
baseline  value. 


Adding  Wew  Branch  Indicators 

Facilitators  should  be  aware  that  managers  of  the  organizational  units  being  aggregated 
frequently  want  to  add  an  indicator  or  indicators  which  apply  to  the  higher-level  organizational 
unit  'e.g.,  branch)  out  which  may  or  may  not  oe  associated  with  the  work  performed  in  the 


lower-level  units  being  aggregated.  For  instance,  in  aggregating  across  work  centers  to  a 
branch,  the  Branch  Chief  may  want  indicators  which  are  unique  to  branch-level  work  in  managing 
the  work  centers.  Such  indicators  might  have  nothing  to  do  with  the  actual  work  done  in  the  work 
centers  themselves.  Such  additional  indicators  may  be  measures  of  activities  that  take  place 
only  at  the  higher  organizational  level  or  they  may  be  activities  that  occur  in  all  of  the 
lower-level  organizations  being  aggregated.  Managers  want  such  higher-level  indicators  because 
they  feel  that  such  indicators  measure  important  parts  of  the  work  of  their  organization  and  are 
needed  to  make  the  aggregation  complete. 

Using  the  example  shown  in  Figure  6,  suppose  the  Branch  Chief  wishes  to  add  a  new  indicator 
that  applies  to  the  branch  and  not  to  the  two  subordinate  work  centers.  To  add  this  new 
indicator,  the  facilitator  leads  the  8ranch  Chief  through  the  usual  process  of  identifying  the 
feasible  best/worst  points  and  the  indifference  point  (or  zone)  to  develop  an  ME  chart  for  this 
one  branch  indicator.  The  feasible  best  is  automatically  assigned  100  effectiveness  points 
(since  this  is  the  only  branch  Indicator)  and  after  some  discussion,  the  feasible  worst  is 
determined  to  be  -100  points.  This  new  ME  chart  is  then  used  to  assess  performance  on  the 
activity  measured. 

In  order  to  aggregate  performance  on  this  branch  Indicator  with  performance  for  the  two  work 
centers,  the  procedures  for  computing  and  applying  the  ACF  must  be  followed.  The  first  step  is 
for  the  Branch  Chief  to  answer  the  question  "How  important  to  the  branch  mission  are-  the 
activities  performed  by  the  two  work  centers  relative  to  the  activity  assessed  by  this  single 
branch  indicator?"  The  facilitator  presents  the  question  to  the  Branch  Chief  in  this  manner: 

YOU  HAVE  SAID  THAT  WORK  CENTER  1  MAKES  THE  GREATEST  CONTRIBUTION  TO  BRANCH 
EFFECTIVENESS  AND  SET  ITS  YALUE  AT  1005  WHILE  WORK  CENTER  2  MAKES  ONLY  HALF  AS  MUCH 
OF  A  CONTRIBUTION  WHICH  IS  SET  AT  505.  BY  COMPARISON,  WHAT  IS  THE  CONTRIBUTION  OF 
THE  ACTIVITY  MEASURED  BY  THE  SINGLE  BRANCH  INDICATOR? 

If  the  Branch  Chief  believes  the  contribution  of  the  activity  measured  by  the  single  branch 
indicator  is  only  one-tenth  as  important  as  the  contribution  of  Work  Center  1,  then  the  MES  for 
the  feasible  best  of  the  branch  indicator  should  be  adjusted  to  .1  of  200,  or  20  MES,  and  branch 
indicator  CESs  should  be  adjusted  with  the  ACF  computed  as  follows: 

MES  Before  Adjustment 

-  =  ACF 

MES  After  Adjustment 

In  this  instance,  the  ACF  is  200/20  =  10.  Thus,  the  CES  of  85  on  the  branch  indicator  is 
adjusted  to  8.5  (85/10)  before  being  added  to  the  corresponding  CESs  from  Work  Centers  1  and  2, 
and  posted  to  the  branch-level  MEP  Feedback  Chart. 

If  the  Branch  Chief  wants  two  or  more  branch-level  Indicators,  the  development  and 
aggregation  process  is  identical  to  that  previously  described.  The  feasible  best/worst  values 
and  indifference  points  (or  zones)  are  developed  for  all  branch  indicators.  The  Indicator 
Weighting  Table  (see  Table  1)  is  used  to  assess  branch  indicators  in  terms  of  maximum 
effectiveness  points,  and  slopes  on  the  ME  charts  are  specified  (as  in  Figure  3).  Again,  the 
3ranch  Chief  evaluates  the  importance  to  the  branch  mission  of  activities  assessed  by  the  branch 
indicators  relative  to  the  contributions  of  Work  Center  1,  whose  importance  to  the  branch  mission 
is  100'S.  Suppose,  as  in  the  earlier  example,  the  contribution  of  each  branch  indicator  is  only 
one-tenth  as  important  as  Work  Center  1  at  1005.  If  so,  the  total  MES  for  all  ME  charts  of  the 
branch  would  be  adjusted  down  to  105  of  the  MES  (200)  of  Work  Center  1.  Then  branch  CESs 
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would  be  adjusted  with  the  ACF  before  being  added  to  the  CESs  of  the  other  work  centers  and 
before  being  posted  to  the  branch-level  MEP  Feedback  Chart. 

The  logical  extension  of  having  aggregated  across  work  centers  to  provide  a  single  measure  of 
the  performance  of  a  branch  is  to  aggregate  branches  to  the  division  and  ultimately  up  the 
chain-of-command  to  larger  organizational  units.  The  process  of  aggregating  across  Increasingly 
larger  organization  units  follows  the  procedures  described  previously.  For  example,  consider 
Figure  7.  Suppose  Branch  A  and  Branch  B  both  have  MGEEM  systems  in  place.  The  Division  Chief, 
the  manager  Immediately  above  the  two  branches,  expresses  an  Interest  in  having  a  single  measure 
of  the  performance  of  the  division.  To  accomplish  this,  aggregation  is  needed. 


aiUNCM  A  BRANCH  3 
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Figure  7.  Aggregating  Across  Branches. 


Aggregating  Across  Branches 


Before  explaining  how  to  aggregate  two  branches  to  a  division,  it  is  perhaps  useful  to 

describe  the  MGEEM  systems  in  place  in  the  branches  and  to  walk  through  the  adjustments  which 

have  been  made  previously  to  aggregate  to  the  branch  level.  After  that,  it  will  be  easy  to 
aggregate  to  the  division  level. 

Figure  7  shows  Branch  A  with  the  two  work  centers  described  earlier  (Figure  6).  For  Branch 
A,  the  MESs  are  200  MES  (100  +  80  +  20)  for  Work  Center  1  and  370  MES  (100  +  90  +  80  +  50  +  50) 

for  Work  Center  2.  Since  the  Branch  Chief  believes  Work  Center  2  Is  only  half  as  important  as 

Work  Center  1,  the  MES  for  Work  Center  2  is  adjusted  to  100  (50*  of  200).  This  value  is  shown  in 
Figure  7  as  the  Adjusted  MES  for  Work  Center  2.  The  ACF  for  Work  Center  2  is  3.7  (370/100).  The 
CES  for  the  3  mission  effectiveness  charts  for  Work  Center  1  is  115  (75,  80,  -40).  To  aggregate 
the  data  across  the  two  work  centers.  Work  Center  2‘s  CES  of  202  (90,  50,  50,  -10,  22)  must  first 
be  adjusted  by  applying  the  ACF.  Finally,  the  CES  for  Branch  A  is  obtained  by  adding  115,  the 
CES  for  Work  Center  1,  to  54.6  (202/3.7),  the  adjusted  CES  for  Work  Center  2,  for  a  sum  of  169.6 
(115  +  54.6).  The  performance  of  Branch  A  of  169.6  can  now  be  posted  to  a  branch-level  MEP 
Feedback  Chart. 

Figure  7  also  shows  Branch  B,  which  has  3  work  centers.  The  three  MES  values  are  230  MES 
(100  +  90  +  40)  for  Work  Center  1,  240  MES  (100  +  50  +  40  +  40  +  10)  for  Work  Center  2,  and  190 
f€S  (100  +  90)  for  Work  Center  3.  For  the  current  measurement  period,  the  CESs  for  the  three 
work  centers,  respecti vely,  are  200,  160,  and  100.  The  Branch  Chief  believes  Work  Centers  1  and 
2,  respectively,  are  only  40*  and  90S  as  important  to  the  branch  as  is  Work  Center  3.  Therefore, 
for  aggregation  purposes,  the  MESs  for  Work  Centers  1  and  2  must  be  reduced  by  these 
percentages.  These  adjusted  MESs  would  be  76  (190  X  .40)  for  Work  Center  1  and  171  (190  X  .90) 
for  Work  Center  2.  The  corresponding  ACFs  would  be  3.0  (230/76)  for  Work  Center  1  and  1.4 
(240/171)  for  Work  Center  2.  To  aggregate  across  work  centers  for  Branch  B,  the  CESs  for  Work 
Centers  1  and  2  must  be  adjusted  by  the  ACF  to  take  into  account  their  lesser  importance  relative 
to  Work  Center  3.  These  adjusted  values  are  67  CES  (200/3.0)  for  Work  Center  1  and  114  CES 
(160/1.4)  for  Work  Center  2.  After  adjustment,  aggregation  for  the  current  period's  performance 
can  be  accomplished  by  adding  the  three  CES  values  (67  +  114  +  100)  to  obtain  a  combined  CES  of 
281.  The  performance  of  Branch  B  (281)  can  now  be  posted  to  a  branch-level  MEP  Feedback  Chart. 

3efore  explaining  how  to  aggregate  the  two  branches  to  the  division  level,  it  is  important  to 
emphasize  again  that  original  ME  charts  and  MEP  Feedback  Charts  which  exist  at  the  work  center 
level  are  not  affected  by  adjustments  for  aggregation  to  the  branch,  division,  or  higher  levels. 
Supervisors  and  workers  in  the  work  centers  will  continue  to  use  these  items  as  initially 
developed. 

The  first  step  in  aggregating  across  branches  to  the  division  level  is  for  the  facilitator  to 
secure  from  the  division  chief  a  judgment  about  the  relative  contributions  of  the  two  branches  to 
the  overall  division  mission.  Using  the  example  shown  in  Figure  7,  the  branch  which  makes  the 
most  important  contribution  Is  ranked  1  and  automatically  given  an  effectiveness  value  of  100*; 
the  other  is  ranked  2  and  scaled  according  to  the  Importance  of  its  contribution  relative  to 
100*.  Of  course,  the  branches  could  be  judged  to  make  equal  contributions.  If  so,  both  would 
receive  an  effectiveness  score  of  100;  and  the  unequal  MESs  for  both  branches,  300  MES  (200  + 
100)  for  Branch  A  and  437  MES  (76  +  171  +  190)  for  Branch  8,  would  have  to  be  made  equal. 
Consequently,  Branch  A  is  adjusted  to  300  because  its  Importance  must  be  made  equal  to  the 
importance  of  Branch  B,  and  its  corresponding  ACF  is  1.46  (437/300).  As  a  result,  the  combined 
division  MES  is  600  (300  +  300). 

To  aggregate  CESs  for  the  two  branches— 169.6  CES  (115  +  54.6)  for  Branch  A  and  281  CES  (67  + 
114  +  100)  for  Branch  3— Branch  3's  CES  must  be  adjusted  with  the  ACF  to  192.5  (281  /1.46)  to  make 
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the  branches'  contributions  equal.  The  two  CES  values  may  then  be  summed  to  362.1  (169.6  + 

192.5).  This  value  may  then  serve  as  the  baseline  for  the  division-level  MEP  Feedback  Chart. 


Suppose,  however,  that  the  Division  Chief  believed  that  Branch  B  was  only  80$  as  important  to 
the  division  mission  as  Branch  A.  Branch  B's  f€S  of  437  would  have  to  be  adjusted  to  240  (300  X 
.80),  and  the  corresponding  ACF  would  be  1.82  (437/240).  As  a  result,  the  division  MES  would  be 
540  (300  +  240).  Before  Branch  B's  CES  (281)  can  be  combined  with  the  CES  of  Branch  A  (169.6), 
it  must  be  adjusted  to  154.4  (281/1.82).  The  two  branch  CESs  may  then  be  summed  to  323.7  (169.3 
+  154.4),  and  this  value  could  serve  as  the  baseline  for  the  division-level  MEP  Feedback  Chart. 


IV.  CONCLUSION 

Thus,  after  developing  ME  charts,  it  is  possible  to  aggregate  mission  effectiveness  scores 
within  and  across  work  centers,  and  within  and  across  increasingly  higher  organizational  levels. 
This  technology  for  “rolling  up“  performance  measures  has  wide  application  and  can  be  very  useful 
to  managers.  A  forthcoming  report  (Weaver,  in  preparation)  will  explain  how  the  MGEEM  system  can 
be  used  to  improve  leadership,  enhance  motivation,  and  continually  improve  work  processes. 
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APPENDIX  8:  FACILITATOR  EXERCISES 


The  following  exercises  are  presented  to  Increase  facilitator  Insight  into  the  implementation 
of  MGEEM.  These  exercises  are  the  result  of  dealing  with  issues  and  problems  encountered  during 
actual  implementation  of  MGEEM  in  over  30  target  organizations  over  a  2-year  period.  They  draw 
upon  material  contained  in  this  guide  on  developing  and  aggregating  ME  charts,  as  well  as 
material  presented  in  the  companion  guide  on  developing  KRAs  and  indicators  (Tuttle  4  Weaver, 
1986).  Because  these  exercises  are  based  on  “real-world*  situations,  some  have  no  single  correct 
answer  but  rather,  a  set  of  possible  solutions. 


Exercise  1:  Degree  of  Control 


Problem.  A  facilitator  Is  developing  indicators  with  Team  B  from  a  target  organization  which 
provides  monthly  qualification  training  for  personnel  in  other  organizations  on  the  base.  The 
target  organization  maintains  records  that  show  which  base  personnel  need  training.  When  an 
individual  is  identified  as  needing  training,  a  training  date  Is  scheduled  and  notification  of 
the  date  is  forwarded  to  the  commander  responsible  for  the  Individual's  attendance.  Often, 
however,  individuals  do  not  show  up  for  scheduled  training.  Their  failure  to  show  up  is  beyond 
the  control  of  the  training  organization  and  Is  almost  always  because  their  workload  is  too  great 
for  their  commander  to  spare  them.  Through  the  MGEEM  process.  Team  B  has  developed  some  good 
indicators  of  performance,  such  as  "average  monthly  test  scores  for  trained  personnel,"  but  the 
manager  of  the  target  organization  says  she  could  plan  better  if  she  had  a  monthly  measure  of 
"training  appointments  kept"  (training  appointments  kept/training  appointments  scheduled  X  100). 
Should  this  measure  be  incorporated  into  the  training  organization's  MGEEM  system  even  though 
they  have  no  control  over  it? 

Suggested  Solution.  It  is  an  error  to  hold  a  target  organization  responsible  for  performance 
over  which  they  have  no  control.  Despite  the  best  efforts  of  these  training  personnel,  factors 
beyond  their  control  cause  attendance  at  their  training  classes  to  fluctuate.  It  would  be 
demotivating  to  give  them  low  scores  on  this  dimension  of  performance  when  they  may  have  done  the 
very  best  they  can. 

This  is  not  to  say  that  the  supervisor  should  not  track  “training  appointments  kept"  and 
whatever  else  she  wishes  to  use  in  her  management  information  system.  A  target  organization’s 
MGEEM  system  and  management  information  system  may  contain  many  or  all  of  the  same  indicators; 
however,  there  are  often  important  differences.  A  key  difference,  of  course,  is  that  MGEEM 
indicators  should  be  only  those  which  measure  activities  over  which  the  target  organization  has 
control.  Also,  MGEEM  indicators  should  encourage  worker  involvement,  be  visible,  be  easy  to 
understand,  and  measure  what  Is  important. 

For  an  excellent  treatment  of  measuring  the  things  that  are  Important,  see  Chapter  VI,  S-l  in 
Peters  (1987).  There  are  good  reasons  for  measuring  only  the  most  important  areas  of  performance 
rather  than  measuring  everything  available.  One  reason  Is  the  smaller  paperwork  burden. 
Undermanned  organizations  with  heavy  workloads  are  highly  resistant  to  devoting  manhours  to 
monitoring  a  complex  measurement  system.  Another  reason  is  that  as  organizations  improve  on 
their  few  but  critical  KRAs,  other  parts  of  their  work  tend  to  improve  as  well.  This  is  true 
because  the  components  of  organizations  are  functionally  related.  Components  must  work  together 
to  accomplish  the  KRAs  and  mission.  To  do  well  on  the  KRAs  almost  always  necessitates  doing  well 
in  even  the  most  indirect  functions  that  support  the  accomplishment  of  KRAs.  This  outcome  will 
not  result,  however,  if  an  important  KRA  is  omitted  from  the  MGEEM  system.  Rather,  the 
measurement  system  will  pull  resources  and  management  attention  away  from  the  unmeasured  area. 


This  is  a  serious  problem  which  should  not  happen  if  Team  A  does  its  job  of  identifying  a 
comprehensive  set  of  KRAs. 


Exercise  2:  No  Indifference  Point 

Problem.  A  facilitator  is  developing  a  measurement  system  with  Team  B  from  a  target 

organization  that  is  a  confinement  facility.  Team  B  wishes  to  have  an  indicator  which  measures 
“number  of  escapes"  (or  “number  of  Instances  of  loss  of  control").  The  feasible  best  Is  "no 
escapes"  and  the  feasible  worst  Is  "one  escape."  When  asked  to  identify  the  indifference  point. 
Team  B  members  say  that  there  is  no  point  on  this  indicator  where  performance  is  neither  good  nor 
bad:  0  escapes  is  good;  1  escape  is  bad.  There  is  no  in-between.  In  using  the  Indicator 

Weighting  Table  to  assess  the  relative  Importance  of  the  Indicators,  Team  B  decides  that  "number 
of  escapes"  is  the  most  Important  of  all  their  Indicators.  "No  escapes"  contributes  more  to  the 
facility's  mission  than  does  the  feasible  best  of  any  other  indicator,  and  “one  escape"  detracts 
more  from  the  mission  than  does  the  feasible  worst  of  any  other  Indicator.  Is  it  possible  to 

develop  an  ME  chart  for  an  Indicator  that  has  no  Indifference  point?  Can  an  ME  chart  be 

developed  for  an  indica^r  that  has  only  two  values,  in  this  case  0  and  1? 

Suggested  Solution.  It  is  possible  to  develop  an  !€  chart  for  an  indicator  with  no 

indifference  point  and  values  of  0  and  1.  Since  this  indicator  is  more  important  than  the 
others,  its  effectiveness  scores  are  -100  for  1  escape  and  100  for  0  escapes.  It  is  misleading, 

however,  to  connect  these  two  points  because  the  resulting  straight  line  will  pass  through  the 

zero  point  on  the  vertical  axis,  giving  the  impression  that  an  indifference  point  exists.  It  is 
probably  best  to  use  a  dashed  straight  line  to  show  there  really  is  no  connecting  line  or 

indifference  point  because  there  can  be  no  1/4,  1/2,  or  3/4  escape.  Figure  B-l  shows  how  this 
chart  should  appear.  The  two  points  are  connected  so  that  all  ME  charts  for  the  facility  will  be 
similar  in  appearance,  but  the  dashed  line  serves  to  remind  organization  members  that  there  are 
really  only  two  levels  of  performance:  0  escapes  with  100  effectiveness  points  and  1  escape  with 
-100  effectiveness  points. 
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Figure  B-l .  Number  of  Escapes. 
Exercise  3:  Exceeding  Feasible  Best/Worst 
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on  a  certain  indicator  is  below  the  feasible  worst  the  organization  said  it  could  do.  It  is, 
therefore,  impossible  to  record  this  level  of  performance  on  the  indicator's  ME  chart.  Does  this 
invalidate  the  chart? 

Suggested  Solution.  Actual  performance  below  a  feasible  worst  or  above  a  feasible  best  on  an 
indicator  invalidates  the  ME  chart,  but  happens  in  almost  every  MGEEM  implementation.  Such  an 
occurrence  should  not,  however,  be  viewed  as  a  serious  problem.  This  simply  requires  that  Team  B 
meet  again  to  consider  a  new  feasible  worst  and  revise  the  scaling  on  the  horizontal  axis  of  the 
indicator  in  question.  They  should  also  review  the  Indicator  Weighting  Table  (retained  from  the 
original  session)  with  the  new  feasible  worst,  for  possible  changes  in  the  relative  importance  of 
other  feasible  bests  and  worsts.  Experience  suggests  that  very  few  changes  usually  result  from 
this  review.  Any  ME  chart  affected  by  this  review  would,  of  course,  simply  be  revised.  An  MGEEM 
system  belongs  to  organization  members  and  they  may  revise  it  at  any  time.  The  purpose  of  the 
system  is  not  to  be  a  rigid  rule-bound  reporting  requirement,  but  a  flexible  self-help  tool  used 
to  work  continuously  toward  improvement. 


Exercise  4:  Horseshoe  Curve 

Problem.  A  certain  test  and  evaluation  work  center  is  developing  an  Indicator  to  monitor  the 
composition  of  its  personnel  force.  The  center  needs  both  college-trained  engineers  and 

field-experienced,  technicians,  and  the  feasible  best  mix  is  about  half  engineers  and  half 
technicians.  In  fact,  the  mix  is  ideal  if  the  percentage  of  either  group  is  between  45%  and 
55%.  As  the  mix  differs  from  this  range,  the  mix  is  still  satisfactory  but  less  so  until  the 
percentage  of  either  group  reaches  40%  or  80%.  8eing  below  40%  or  above  80%  has  negative  effects 
on  the  mission,  increasing  in  negative  effect  up  to  mixes  of  10%  and  90%  which  are  the  feasible 

worsts  that  could  occur.  Compared  with  other  indicators,  the  feasible  best  earns  50 

effectiveness  points  and  both  feasible  worsts  incur  -90  effectiveness  points.  It  is  possible  to 

create  an  ME  curve  for  this  indicator? 

Suggested  Solution.  Figure  B-2  shows  that  the  ME  chart  for  this  situation  is 
horseshoe-shaped  because  there  are  two  points  of  feasible  worst,  two  indifference  points,  and  one 
zone  of  feasible  best.  The  effectiveness  points  for  the  feasible  worsts  are  both  -90  and  the 
feasible  best  zone  is  at  50  effectiveness  points.  If  it  reflects  the  judgment  of  Team  B, 
straight  lines  can  connect  these  points.  Of  course,  the  feasible  worsts  are  not  required  to  have 
the  same  effectiveness  points  if  having  far  too  many  of  one  group  is  worse  than  having  far  too 
many  of  the  other,  'lor  is  it  required  that  the  connecting  lines  be  straight.  In  any  case,  this 
example  demonstrates  that  curves  do  not  always  run  from  lower-left  to  upper-right. 
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rlgure  3-2.  Percent  of  Engineers. 
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Exercise  5;  Aggregating  to  Branch  Level 


Problem.  A  certain  Materiel,  Storage,  and  Distribution  (MSAD)  Branch  in  a  Supply  Squadron 
recently  implemented  an  MGEEM  system.  The  MSAD  Branch  has  4  sections,  which  the  Branch  Chief 
believes  contribute  equally  to  the  accomplishment  of  the  branch  mission.  The  MESs  and  CESs  for 
the  4  sections  and  for  one  branch  indicator  are  shown  in  Table  B-l. 


Table  B-l.  Section  and  Branch  Performance  Data 


Materiel, 

Storage  and  Distribution  Branch 

Branch 

Indicator 

Receiving 

Storage 

and 

Issue 

Pick  up 
and 

Del  1  very 

Inspection 

No.  of  Indicators 

5 

7 

9 

11 

1 

MES 

450 

585 

734 

85 

CES 

423 

531 

387 

631 

65 

Section  chiefs  use  these  data  to  evaluate  how  well  the  sections  are  doing.  The  CESs  for  each 
section  (423,  531,  387,  and  631)  are  each  plotted  on  MEP  Feedback  Charts  for  monitoring  by  their 
section  chiefs.  The  one  branch-level  indicator  CES  of  65  is  also  monitored  by  the  Branch  Chief 
with  a  MEP  Feedback  Chart.  Since  the  sections  are  of  equal  importance,  the  overall  effectiveness 
of  the  MSAD  Branch  is  determined  by  summing  the  function  CESs  to  2,037  (423  +  531  +  387  +  631  + 
65)  effectiveness  points.  The  Branch  Chief  plots  the  "rolled  up"  branch  CES  of  2,037  on  a  MEP 
Feedback  Chart,  which  is  used  to  monitor  overall  branch  performance.  Is  there  anything  wrong 
with  this  approach  to  measuring  the  performance  of  the  sections  and  aggregating  to  the  branch? 

Suggested  Solution.  The  procedure  used  to  evaluate  the  performance  of  the  sections  within 
the  MS  AD  Branch  Is  correct.  Within  each  section,  ME  charts  translate  performance  on  indicators 
into  effectiveness  scores  which  may  be  summed  to  provide  section  CESs.  Changes  in  CESs  for  each 
section,  for  the  one  branch  indicator,  and  for  the  overall  branch  are  correctly  shown  on  MEP 
Feedback  Charts. 

However,  aggregation  across  sections  by  simply  adding  section  CESs  and  ME Ss  is  an  error 
despite  the  fact  that  the  Branch  Chief  has  asserted  that  the  sections  contribute  equally  to  the 
branch  mission.  If  the  sections  are  to  contribute  equally,  the  MESs  must  also  contribute 
equally— not  unequally  as  they  currently  do  with  MESs  of  450,  585,  600,  and  734.  This  is  the 
"unequal  contribution  problem"  discussed  In  Section  III.  An  ACF  must  be  computed  and  applied  to 
make  these  MESs  e-  .  In  keeping  with  the  methods  explained  in  this  report,  use  of  the  smallest 
MES  (i.e. ,  adjustment  downward)  is  probably  most  convenient.  In  the  case  of  this  MSAD  Branch, 
this  number  is  450  for  Receiving;  therefore,  all  section  MESs  are  changed  to  450. 


The  next  step  to  correct  the  "unequal  contribution  problem"  Is  to  use  the  ACF  to  modify  the 
unequal  CESs  to  reflect  the  equal  contributions  of  the  sections.  This  Involves  computing  the  ACF 
(MES  before  adjustment/MES  after  adjustment)  for  each  section.  Ho  modification  is  needed  for  the 
Receiving  section  since  its  MES  was  selected  as  the  common  number.  That  is,  the  Receiving 
Section's  MES  remains  450.  In  any  case,  the  ACF  for  the  other  3  sections  are  1.30  (585/450)  for 
Storage  and  Issue,  1.33  (600/450)  for  Pickup  and  Delivery,  and  1.63  (734/450)  for  Inspection. 
The  CESs  for  the  4  sections  are  modified  with  the  ACF  as  follows:  The  531  CES  for  Storage  and 
Issue  is  adjusted  to  408  CES  (531/1.3);  and  the  other  adjustments  are  387  CES  to  290  CES 
(387/1  .33)  for  Pickup  and  Delivery,  and  631  to  387  CES  (631/1.63)  for  Inspection.  It  would  be 
appropriate  for  oectio.,*  to  plot  an  adjusted  CES  for  their  MEP  Feedback  Charts,  and  the  branch 


CES  of  1,535  based  on  adjusted  section  CESs  (450  +  408  +  290  +  387)  could  be  plotted  on  a  MEP 
Feedback  Chart  to  track  periodic  changes  In  the  performance  of  the  branch.  However,  this 
aggregation,  while  correct,  would  fall  to  take  into  consideration  the  single  branch  indicator 
shown  in  the  rightmost  column  of  Table  B-l. 

In  order  to  add  the  branch  Indicator,  the  facilitator  would  ask  the  Branch  Chief  for  a 
judgment  as  to  the  contribution  to  the  branch  mission  of  the  activity  measured  by  the  branch 
Indicator.  The  facilitator  says,  "IF  THE  SECTION  CONTRIBUTIONS  ARE  EQUAL  AT  100%,  WHAT  IS  THE 
PERCENTAGE  CONTRIBUTED  BY  THE  ACTIVITY  MEASURED  BY  THE  BRANCH  INDICATOR?"  Suppose  the  Branch 
Chief  says  "15%.“  This  means  that  since  the  sections  contribute  equal  MESs  of  450,  the  one 
branch  indicator  can  contribute  for  its  MES  only  1 5*  of  450,  or  67.5  (450  X  .15).  Thus,  the 
branch  Indicator's  MES  is  67.5. 

Next,  the  branch  Indicator's  CES  must  be  modified  with  the  ACF,  MES  before  adjustment/MES 
after  adjustment.  This  ACF  is  1  .26  (85/67.5).  Therefore,  the  branch  Indicator  CES  of  65  Is 
modified  to  51.6  (65/1.26).  Thus,  the  correct  aggregation  for  the  branch,  including  the  4 
sections  and  the  one  branch  indicator,  results  In  an  MSSD  Branch  CES  of  1,586.6  (450  +  408  +  290 
+  387  +  51.6),  which  can  be  plotted  on  a  MEP  Feedback  Chart  to  track  periodic  changes  In  the 
performance  of  the  branch  on  all  indicators. 


Exercise  6:  Feedback  Alternatives 


Problem.  Consider  the  case  of  the  Receiving  Section  of  the  MSSD  Branch  discussed  in  Exercise 
5.  The  section's  CES  and  MES  for  5  indicators  were  423  and  450,  respectively.  Instead  of 
monitoring  CESs  over  time  with  a  MEP  Feedback  Chart,  the  Section  Chief  decides  to  use  a  different 
procedure.  The  Section  Chief  reasons  that  the  MES  of  450  represents  the  effectiveness  points  of 
the  feasible  best  the  section  can  do,  and  the  CES  of  423  Is  the  section's  current-month 
performance.  Thus,  the  section's  current  performance  of  423  Is  94%  (423/450  X  100)  of  the 
feasible  best  that  can  be  done  (450).  The  Section  Chief  labels  this  result  as  the  "Percent  of 
Maximum  Achievable"  and  monitors  it  over  time,  referring  to  It  in  monthly  feedback  sessions  with 
workers.  Is  there  anything  wrong  with  using  “Percent  of  Maximum  Achievable"  instead  of  a  MEP 
Feedback  Chart? 

Suggested  Solution.  There  are  two  serious  problems  associated  with  use  of  the  "Percent  of 
Maximum  Achievable."  The  first  problem  has  to  do  with  the  arithmetic  Involved  in  calculating  the 
percentage.  This  percentage  is  only  meaningfully  Interpretable  for  CES  values  that  are  zero  or 
positive.  Suppose,  for  Instance,  that  the  CES  for  the  Receiving  Section  falls  below  0 
effectiveness  points;  that  is,  the  sum  of  the  effectiveness  points  for  all  5  ME  charts  is  less 
than  zero,  say  -25.  The  calculation  for  the  "Percent  of  Maximum  Achievable"  would  be  -25/450  X 
100,  or  -5.6%.  In  other  words,  for  this  month  the  section's  performance  is  -5.6%  of  the  best  it 
can  be.  This  negative  percentage  Is  meaningless.  An  equally  unsatisfactory  solution  is  to  use 
the  other  end  of  the  scale  to  compute  "Percent  of  Minimum  Achievable."  If  the  minimum  achievable 
were,  say,  -400,  the  calculation  would  be  -25/-400  X  100,  or  62.5%.  The  answer  is  interpretable, 
but  not  reasonable  as  a  measurement  device.  It  would  sound  absurd  to  say,  "The  Receiving  Section 
is  doing  62.5%  of  the  worst  It  can  do." 

Another  reason  to  use  the  MEP  Feedback  Chart  Instead  of  the  “Percent  of  Maximum  Achievable" 
Is  that  graphics  which  Identify  and  focus  on  the  maximum  performance  achievable,  the  MES,  may 
tempt  some  managers  to  drive  the  organization  to  higher  and  higher  performance.  Indeed, 
continual  Improvement  is  the  goal  of  the  MGEEM,  but  it  should  be  achieved  through  increased 
worker  motivation  stimulated  by  feedback  and  the  removal  of  constraints.  The  MGEEM  process  will 
soon  fail  if  it  is  used  as  a  tool  for  coercion  and  control.  Fortunately,  most  Air  Force  managers 
are  aware  that  autocratic,  self-serving  styles  of  management  result,  at  best,  in  only  short-term, 
imp  rovements. 
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Exercise  7:  Interdependent  Indicators 


Problem.  While  developing  ME  charts  for  an  organization,  a  facilitator  overhears  several 
members  of  Team  B  discussing  the  construction  of  one  particular  chart.  One  member  of  the  team 
says,  "But,  how  well  we  do  on  that  indicator  depends  on  how  well  we  are  doing  on  a  different 
indicator."  The  facilitator  explores  the  situation  further  and  learns  that  team  members  think 
that  the  two  indicators  are  related  in  such  a  manner  that  If  the  organization  is  performing 
poorly  on  one  of  the  indicators  (Indicator  #3),  it  does  not  make  much  difference  how  they  are 
doing  on  the  other  (Indicator  #1). 

As  a  remedy,  the  facilitator  suggests  that  the  team  construct  at  least  two  different  ME 
charts  for  Indicator  #1  whose  curves  will  differ  depending  on  the  level  of  Indicator  #3.  How  can 
the  two  charts  be  constructed  and  how  would  they  be  integrated  into  the  work  center's  existing 
set  of  ME  charts? 

Suggested  Solution.  One  possible  answer  to  this  problem  is  that  there  may  be  no  real 
problem;  that  is,  it  may  be  that  the  conditions  under  which  one  indicator  becomes  dependent  upon 
the  other  seldom,  if  ever,  occur  In  the  real-world  situation.  Such  is  the  case  in  Figure  B-3, 
where  the  ME  chart  for  an  indicator  shows  a  wide  range  of  acceptable  performance  (from  the  dashed 
line  upward  to  the  top  right  corner).  Though  there  is  a  possibility  that  performance  on  this 
indicator  might  reach  the  "worst"  level,  it  is  very  unlikely  to  happen.  If  and  when  it  does,  the 
measurement  of  mission  effectiveness  for  that  Indicator  would  have  to  be  handled  on  an  ad  hoc 
basis  until  the  crisis  passed.  If  conditions  persist  more  than  one  or  two  measurement  periods. 
Team  8  should  reconvene  to  reassess  the  worst  and  best  possible  levels  for  the  interacting 
indicators. 


If  there  is  a  real  problem,  however,  and  the  conditions  under  which  one  indicator's  mission 
effectiveness  is  dependent  on  the  performance  level  of  another  indicator  occur  fairly  often,  then 
an  approach  similar  to  the  one  shown  in  Figure  9-4  could  be  taken.  Figure  B-4  shows  the 
situation  where  the  shape  of  the  ME  chart  for  one  indicator  (Indicator  #1)  is  dependent  on  the 
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Figure  B-4.  Mission  Effectiveness  Charts  Interactions. 
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level  of  performance  on  another  indicator  (Indicator  #3).  Figure  8-4 ( a )  shows  the  ME  chart  for 
Indicator  #3.  In  this  case,  it  might  be  useful  to  have  two  ME  charts  for  Indicator  #1 ,  so  that 
when  Indicator  #3  performance  is  in  the  low  performance  level  part  of  its  ME  chart,  then  the 

chart  shown  in  Figure  B-4(b)  applies,  but  when  Indicator  #3  performance  is  at  the  higher 
performance  level,  the  chart  in  Figure  B-4(c)  applies.  (Mote:  For  this  example,  only  2 

alternative  ME  charts  are  used,  but  additional  levels  for  Indicator  #1  are  permissible.) 

Note  that  the  values  on  the  horizontal  axis  do  not  change  for  the  dependent  indicator 

(Indicator  #1);  however,  the  slopes  of  the  ME  chart  curves  change,  as  do  the  effectiveness  scores 
associated  with  performing  at  the  feasible  worst  and  best  levels.  In  the  present  example,  the 

feasible  best  and  worst  are  worth  30  and  -30  in  Figure  B-4(b)  but  shift  to  100  and  -100, 

respectively,  in  Figure  B-4(c).  In  order  to  accommodate  this  change,  the  Team  B  members  would 
have  to  reallocate  the  change  in  total  effectiveness  points  to  other  indicators  for  the  work 

center.  For  example,  suppose  the  work  center  originally  had  three  indicators  whose  total 
positive  effectiveness  score  was  250  (from  Indicators  #1,  #2,  and  #3  which  have  a  maximum 
positive  effectiveness  score  of  100,  90,  and  60  points,  respectively).  If  Indicator  #l's  maximum 
positive  effectiveness  score  of  100  is  dependent  on  the  level  of  performance  on  Indicator  #3  and 
Indicator  #3  is  at  its  low  performance  level,  then  the  maximum  effectiveness  score  for  Indicator 
#1  becomes  20  points.  Therefore,  80  points  (100-20)  would  have  to  be  reapportioned  among 
Indicators  #2  and  #3  in  the  same  ratio  as  their  original  contributions  to  overall  effectiveness. 

The  relative  contributions  compared  to  the  original  most  important  indicator  (Indicator  #3) 
were  90S  for  Indicator  #2  and  60S  for  Indicator  #1.  Therefore,  in  dividing  up  the  80  points. 
Indicator  #2  gets  90/(90  +  60)  *  .6  X  80  =  48. effectiveness  points  added  to  its  current  maximum 

of  90,  to  give  138  points.  Indicator  #1  gets  60/(90  +  60)  =  .4  X  60  3  32  effectiveness  points 

added  to  its  maximum  of  60,  giving  92  points. 

When  Indicator  #3  is  at  its  higher  performance  level,  no  change  in  the  original  250  total 
effectiveness  points  is  necessary.  In  any  case,  however  (whether  Indicator  #3  is  at  its  high 
performance  level  or  not),  only  3  indicators  are  used  and  they  still  have  a  total  positive 
effectiveness  score  of  250  (20  +  138  +  92). 

The  maximum  negative  score  would  be  reallocated  in  a  similar  manner. 
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MANAGING  WITH  MGEEM 

Charles  N.  Weaver,  Ph.D. 

Air  Force  Human  Resources  Laboratory  (AFHRL) 

Air  Force  Regulation  25-5  (May,  1988)  requires  that  every  Air  Force 
functional  area  have  an  adequate  management  information  system  (MIS)  and  this 
is  reviewed  during  the  functional  review  process.  The  Methodology  for 
Generating  Efficiency  and  Effectiveness  Measures  (MGEEM)  is  incorporated  into 
the  Performance  Measures  Document  (PMD)  of  this  regulation  (Chapter  6)  as  the 
recommended  measurement  component.  The  MGEEM  is  a  comprehensive  MIS  which 
makes  possible  improved  leadership,  enhanced  motivation,  and  the  continual 
improvement  of  the  processes  by  which  work  is  conducted.  The  MGEEM  provides  a 
framework  for  implementing  the  philosophy  and  principles  of  Total  Quality 
Management  (TQM). 

HRL  has  published  technical  reports  (Tuttle  &  Weaver,  1986;  Weaver  & 
Looper,  1989)  explaining  hew  to  build  an  MGEEM  system.  There  is  need, 
however,  for  guidance  to  workers,  supervisors,  and  commanders  on  how  to  use  an 
MGEEM  system  when  one  exists  in  their  organization.  HRL  is  completing  a 
technical  report  which  will  contain  this  guidance.  This  paper  is  an  overview 
of  this  information. 

For  purposes  of  illustration,  assume  a  certain  Security  Police  work  center 
has  an  MGEEM  system  with  two  key  result  areas  (KRAs),  "to  protect  base 
resources"  and  "to  document  security  actions."  For  each  KRA,  there  is  one 
indicator.  For  protecting  base  resources  the  indicator  is  "number  of 
reportable  incidents"  and  for  documenting  security  actions  the  indicator  is 
"percent  of  reports  completed  on  time."  (Number  of  reportable  incidents  is 
the  absolute  number  and  percent  of  reports  completed  on  time  is  the  number  of 
reports  completed  on  time  divided  by  the  number  of  reports  required  x  100). 
Mission  Effectiveness  (ME)  charts  for  these  two  indicators  are  shown  in 
Figure  1 . 


Figure  1.  ME  Charts 


MGEEM  as  an  MIS.  The  MGEEM  provides  much  useful  information  about  the 
operation  of  the  work  tenter.  For  instance,  KRAs  show  that  the  mission  of  the 
work  center  is  to  protect  base  resources  and  document  security  actions. 
Indicators  measure  the  work  center's  performance  in  accomplishing  the  KRAs. 
There  is  one  ME  chart  for  each  indicator  showing  the  relationship  between 
levels  of  performance  on  indicators  (on  the  horizontal  axis)  and  the  work 
center's  overall  mission  effectiveness  (on  the  vertical  axis).  Slopes  on  ME 
charts  show  the  importance  of  Indicators:  flat  slopes  mean  indicators  are 
less  important  than  indicators  with  steep  slopes.  For  instance,  the  slopes  on 
the  ME  charts  in  Figure  1  indicate  that  number  of  Incidents  is  more  Important 
to  mission  effectiveness  than  reports  completed  on  time.  When  the  work  center 
performs  as  well  as  feasible  on  both  indicators,  "no  incidents"  has  a  +100 
impact  on  mission  effectiveness  while  "completing  95%  of  reports  on  time"  has 
only  a  +20  impact  on  mission.  Thus,  comparing  the  slopes  on  the  two  ME  charts 
shows  policy  about  the  importance  of  the  two  indicators:  controlling 
incidents  is  more  important  than  completing  reports  on  time.  This  is  not  to 
say  that  the  work  center  can  neglect  completing  reports  on  time.  Completing 
less  than  85%  of  reports  on  time  has  a  negative  impact  on  mission 
effectiveness  but  it  is  not  so  serious  as  a  high  number  of  incidents. 
Furthermore,  when  the  work  center  completes  as  many  reports  on  time  as 
feasible  (95%)  it  only  has  a  moderately  positive  effect  on  mission 
effectiveness  (+20). 

As  the  work  center  strives  to  improve,  ME  charts  provide  feedback  on  how 
successful  the  work  center  has  been.  Each  month  the  results  of  performance  on 
the  indicators  are  recorded  on  ME  charts  and  provided  to  the  supervisor  and 
workers  and  posted  on  bulletin  boards  in  the  work  centers  for  all  to  see. 
Suppose,  for  instance,  that  last  month's  performance  on  the  two  indicators  in 
Figure  1  was  50  incidents  and  80%  of  reports  completed  on  time.  These  results 
are  posted  (in  Figure  2  with  a  letter  representing  the  month.  Here  the  letter 
is  0  for  October.)  on  ME  charts  and  provided  as  feedback  to  the  supervisor, 
NCOs,  and  enlisted  personnel.  These  results  show  that  the  work  center  is 


Figure  2.  One  Month's  Performance 
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doing  well  on  the  most  important  aspect  of  its  business,  protecting  base 
resources.  A  50  on  number  of  incidents  (on  the  horizontal  axis)  translates 
into  a  +45  effectiveness  point  (on  the  vertical  axis)  and  shows  that  there  is 
room  for  Improvement  but,  in  general,  that  things  are  going  well  on  this 
i ndicator. 


On  the  second  indicator,  however,  80%  of  reports  completed  on  time  shows 
that  the  second  important  aspect  of  business,  documenting  security  actions,  is 
not  being  accomplished  very  well.  Completing  80%  of  reports  (on  the 
horizontal  axis)  has  a  negative  impact  of  about  -10  on  mission  effectiveness 
(on  the  vertical  axis).  With  this  Information,  the  supervisor  and  workers 
should  discuss,  evaluate,  and  employ  initiatives  for  improvement.  At  the  end 
of  the  month  following  such  initiatives,  performance  results  are  again  added 
to  the  ME  charts  and  provided  to  the  supervisor,  NCOs,  and  enlisted 
personnel.  The  new  results  are  indicated  on  the  ME  charts  with  another 
letter,  here  N  for  November,  so  that  current  period  performance  can  be 
compared  against  the  performance  of  the  previous  period(s).  With  an  MGEEM 
system  in  place  the  supervisor  can  monitor  work  center  performance  from 
month- to-month  to  track  the  results  of  management  initiatives  and  other 
influences,  such  as  inspections  and  deployments.  Of  course,  efficiency 
(outputs  divided  by  inputs)  should  also  be  considered  in  managing  work  centers. 


Another  part  of  the  MGEEM  MIS  is  that  a  summary  of  the  performance  of  a 
work  center  can  be  rolled  up  monthly  to  the  next  higher  level  of  supervision 
on  an  MEP  chart.  This  Information  provides  a  quick  overview  of  work  center 
performance.  Overall  work  center  effectiveness  (the  sum  of  the  current  period 
vertical  axis  effectiveness  scores  on  each  ME  chart)  for  a  given  period  is 
simply  being  plotted  as  shown  In  Figure  3.  In  Figure  3  suppose  the  summed  or 
rolled  up  effectiveness  scores  are  35  for  October  and  65  for  November.  The 
scale  of  the  vertical  axis  is  adjusted  to  accommodate  the  feasible  range  of 
rolled  up  values.  The  vertical  axis  may  contain  negative  values. 
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Figure  3.  Example  MEP  Chart 
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Improved  Leadership.  Before  an  MGEEM  system  becomes  operational  (or  modified 
after  a  period  of  use)  it  is  coordinated  with  the  squadron  commander.  This 
coordination  preserves  traditional  military  prerogatives  but  should  also 
demonstrate  the  squadon  commander's  respect  for  the  work  center  supervisor's 
judgment  and  experience.  The  squadron  commander  should  assure  that 
coordination  on  the  MGEEM  system  strengthens  the  bond  with  his/her  work  center 
supervisors.  The  squadron  commander  should  be  a  model  of  good  leadership  to 
work  center  supervisors  and  NCOs.  Good  leadership  for  all  levels  is  not  based 
on  micromanagement  with  quotas,  suspense  dates,  and  fear.  It  should  be  based 
on  a  coaching,  team  building,  teaching  relationship. 

After  the  MGEEM  system  has  been  approved,  it  provides  the  supervisor  with 
a  number  of  management  tools.  For  instance,  consider  the  ME  charts  in  Figure 
2  where  the  letters  0  represent  current  performance.  Is  it  better  to  manage 
the  work  center  to  maintain  or  reduce  "number  of  incidents"  or  to  increase 
"reports  completed  on  time?"  The  slope  of  the  charts  shows  that  the  greatest 
payoff  to  effectiveness  is  in  maintaining  or  reducing  incidents.  Reports 
cannot  be  neglected,  but  the  ME  charts  show  that  they  are  not  so  important  as 
incidents  to  the  mission.  In  addition,  what  happens  when  the  month-to-month 
results  posted  on  an  ME  chart  go  in  the  wrong  direction?  This,  of  course, 
means  there  is  a  problem  somewhere.  Work  center  morale  and  cohesion  is 
improved  when  supervisors  solicit  ideas  about  the  problem  from  all  work  center 
personnel  in  "feedback  sessions"  (described  below).  Instead  of  seeking  to 
blame  someone  when  a  problem  is  detected,  supervisors  should  focus  on 
improving  the  processes  by  which  work  is  conducted.  Supervisors  must  listen. 
They  should  seek  to  solve  problems  together  with  workers.  They  should  seek 
more  data  and  analyses  about  the  problem,  depending  less  on  judgment  and 
guesswork.  They  shouldn't  be  hurried,  taking  more  time  for  precision  and 
care.  They  should  solicit  worker's  ideas  about  constraints  to  higher 
performance.  Supervisors  should  say,  "What  can  I  do  to  help  you  achieve 
excellence?  What  can  I  do  to  help  you  lead  a  better  work  center?" 

Enhanced  Motivation.  The  most  important  motivational  feature  of  the  MGEEM 
system  is  monthly  "feedback  sessions."  As  soon  as  current  monthly  performance 
results  are  posted  on  ME  charts,  the  charts  should  be  provided  to  the  work 
center  supervisor.  As  soon  as  the  charts  are  available  they  should  be 
distributed  at  monthly  meetings  of  key  work  center  personnel.  The  supervisor 
and  NCOs  should  be  present  throughout  the  meeting,  along  with  other  members  of 
the  work  center  and  customers  and  suppliers,  as  needed.  (In  a  large  work 
center  other  personnel  should  attend  on  a  rotating  basis,  say  every  other 
month.)  Everyone  should  have  frequent  opportunities  to  attend  and  get  in 
their  "2  cents  worth."  No  member  of  the  work  center  should  be  left  out  of  the 
feedback  sessions.  At  these  meetings  the  supervisor  should  encourage  feedback 
about  how  to  improve  work  center  performance  as  measured  on  the  ME  charts. 

The  supervisors  and  NCOs  should  listen  to  the  reactions  and  suggestions  of 
enlisted  personnel.  Enlisted  personnel  should  be  asked,  "What  keeps  us  from 
doing  a  better  job?  What  holds  us  back?"  Remember  that  the  work  center  is  a 
team.  In  the  feedback  sessions,  team  building  and  teamwork  should  be 
encouraged.  Supervisors  should  not  look  for  someone  to  blame  for  a 
performance  problem,  but  should  seek  to  figure  out  ways  to  improve  the  work 
processes  involved.  Supervisors  must  understand  that  the  vast  majority  of 
personnel  want  to  do  a  good  job.  They  want  to  take  pride  in  their  work. 
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Experience  shows  that' some  inadequacy  in  the  management  systems  is  usually  the 
real  cause  of  performance  problems.  For  instance,  is  everyone  adequately 
trained?  Is  there  enough  time  to  do  a  good  job?  Are  supplies  and  materials 
appropriate  and  available?  Are  instructions  clear?  Is  supervision  supportive 
and  encouraging?  Has  time  been  taken  to  build  morale  and  espirit  de  corps? 

Do  managers  at  every  level  take  time  to  listen  to  their  subordinates?  Is 
self-improvement  encouraged  among  subordinates? 

Continually  Improving  the  Way  Work  is  Conducted.  In  feedback  sessions  the 
supervisor  should  lead  a  discussion  of  how  to  make  things  better:  "How  can  we 
improve  on  our  ME  charts?  What's  holding  us  back  from  greater  excellence?" 

As  explained  above,  experience  shows  that  most  performance  problems  result 
from  inadequacies  in  the  way  the  work  center  does  its  business,  not  from 
people  unwilling  to  work  hard.  It  is,  therefore,  almost  always  a  waste  of 
time  to  expect  to  get  lasting  gains  in  performing  by  exhorting  subordinates  to 
work  harder.  Most  personnel  are  already  working  at  their  limit.  Many  already 
work  10-12  hours  a  day.  To  ask  them  to  work  more  hours  is  an  insult  to  them 
and  an  unnecessary  burden  on  their  families.  Rather  than  asking  them  to  work 
harder,  supervisors  must  accept  responsibility  for  simplifying  and 
streamlining  the  way  the  work  is  done.  It  is  the  manager's  responsibility  to 
figure  out  how  her/his  people  can  work  smarter,  not  harder,  to  accomplish  the 
mission.  The  key  question  is,  of  course,  how  do  we  work  smarter  rather  than 
harder? 

Evidence  shows  that  working  smarter  rather  than  harder  is  the  result  of  a 
certain  management  philosophy  and  a  number  specific  initiatives.  The 
philosophy,  already  identified  above,  is  that  rather  than  exhorting 
subordinates  to  work  harder  managers  accept  the  responsibility  to  improve  the 
way  work  is  conducted.  There  are  a  number  of  specific  initiatives  to  improve 
the  way  work  is  conducted.  First,  supervisors  should  accept  MGEEM  ME  charts 
as  measures  of  how  well  the  work  center  mission  is  being  accomplished.  They 
hold  should  monthly  feedback  sessions  with  subordinates  to  study  the  charts 
and  "brainstorm"  ideas  on  how  to  improve  performance.  They  should  use  simple 
statistical  tools  to  search  for  ways  to  improve  work  processes.  (For 
discussions  of  these  tools,  see  Ishikawa,  1982,  and  Brassard,  1988.) 
Subordinate  input  should  be  valued  and  encouraged.  The  focus  should  not  be  on 
finding  someone  to  blame  for  problems  but  on  how  to  improve  the  way  business 
is  conducted.  Second,  squadron  commanders  and  supervisors  should  be  willing 
to  experiment  with  new  ideas  that  have  potential  for  simplifying  and 
streamlining  the  work.  They  authorize  the  implementation  of  new  ideas  on  a 
test  basis  and  evaluate  the  impact  of  the  test  with  their  ME  charts.  They  are 
willing  to  "write  off"  several  failures  to  get  to  one  winning  idea  that 
significantly  improves  work  center  performance.  They  should,  of  course, 
coordinate  the  implementation  of  new  ideas  and  utilize  existing  Air  Force 
change  programs,  including  the  Model  Installations  Program  (MIP),  the 
Suggestion  Program  (SUG) ,  and  Management  Decision  Package  (developed  during 
functional  reviews).  Managers  should  read  and  reread  any  books  by  TQM  experts 
such  as  Deming,  Juran,  and  Peters. 

In  an  organizational  climate  of  harmony  and  respect  the  MGEEM  provides  an 
MIS  which  makes  possible  improved  leadership,  enhanced  motivation,  and  the 
continual  improvement  of  the  processes  by  which  work  is  conducted.  Experience 
shows  that  use  of  the  MGEEM  as  the  PMD  MIS  improves  morale,  improves  quality, 
and  increases  organizational  effectiveness. 
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IMPLEMENTING  TQM  WITH  MGEEM 
Charles  N.  Weaver,  PhD 

Air  Force  Human  Resources  Laboratory  (AFHRL) 

There  is  widespread  interest  in  the  Department  of  Defense  (DoD)  in 
implementing  Total  Quality  Management  (TQM).  There  are  a  number  of  different 
definitions  of  TQM,  but  it  is  almost  universally  agreed  that  its  essence  is 
embodied  in  the  philosophy  and  teachings  of  W.  Edwards  Deming.  Dr  Deming' s 
wisdom  is  available  in  his  4-day  seminar,  nis  books  (e.g.,  Deming,  1986),  and 
video  tapes.  Other  books  on  Dr  Deming  and  his  wisdom  are  by  Gitlow  and  Gitlow 
(1987),  Killian  (1988),  Mann  (1985),  Sherkenback  (1988),  and  Walton  (1986). 
Commanders  and  managers  tasked  to  implement  TQM  in  their  organizations  often 
report,  however,  that  implementing  TQM  is  far  more  difficult  than 
understanding  it.  In  our  experience,  implementing  TQM  is  almost  always 
characterized  by  uncertainty,  confusion,  false  starts,  management's  delegation 
of  its  TQM  responsibilities,  a  burst  of  activity  to  "fill  a  square"  and  then 
back  to  business  as  usual,  and  money  virtually  wasted  on  contractors  and 
consultants. 

To  our  knowledge,  efforts  to  implement  TQM  follow  two  basic  approaches. 

The  first  is  the  use  of  a  high-level  steering  committee  which  originates  and 
directs  problem-solving  teams,  commonly  called  process  action  teams  (PATS). 

The  second  involves  (a)  management  in  authority  teaching  TQM  philosophy  so 
that  a  critical  mass  of  employees  becomes  involved  in  the  necessary  cultural 
transformation  and  (b)  hiring  a  master  statistician  who  provides  statistical 
assistance  throughout  the  organization.  Under  ideal  conditions  these 
approaches  produce  favorable  results,  but  experience  shows  that  conditions  are 
usually  far  from  ideal.  For  instance,  a  common  result  of  the  steering 
committee/PAT  approach  is  that  it  is  an  easy  way  for  commanders  and  managers 
to  avoid  their  TQM  responsibilities.  They  merely  set  up  a  few  PATs  and  return 
to  work  as  usual.  They  "fill  squares"  by  pointing  to  the  number  of  PATs  in 
place,  the  number  of  people  who  get  PAT  training,  the  number  of  PAT  reports 
provided,  and  the  resources  spent  on  their  TQM  effort.  It  is  true  that  when 
the  steering  committee/PAT  approach  is  used,  processes  are  improved  but  the 
real  problem  is  that  no  attempt  is  usually  made  to  bring  about  the 
transformation  called  for  by  Dr  Deming. 

There  are  other  improper  uses  of  the  steering  committee/PAT  approach. 
Steering  committees  almost  always  set  up  PATs  based  on  solely  on  judgment, 
usually  in  areas  where  they  hear  the  most  complaints.  This  approach  will 
often  be  successful  in  improving  processes  until  the  obvious  problem  areas  are 
improved.  But  after  the  obvious  problem  areas  are  improved,  what  next?  Where 
are  new  PATs  established?  Beyond  the  very  serious  few,  steering  committees 
seldom  know  which  other  areas  have  problems  that  are  serious  enough  to  deserve 
the  attention  and  resources  of  a  PAT.  Steering  committees  seldom  have  a 
systematic  approach  to  the  establishment  of  PATs.  Ideally,  the  most  serious 
problem  areas  should  be  addressed  first.  Often  steering  committees  will 
establish  PATS  everywhere  (even  where  they  are  not  needed)  or  delegate  the 
decisions  about  where  to  establish  them  to  the  judgment  of  lower-level 
personnel.  In  such  cases  a  memorandum  goes  out,  "Who  wants  to  be  on  a  PAT  in 
such-and-srch  division?  Volunteer  please  contact  the  division  chief  by 
such-and-such  date."  Volunteers  are  then  given  training  in  the  Deming  cycle 


and  the  7  tools  and  turned  loose  to  work  any  problems  they  wish.  No  guidance 
is  provided  concerning  the  importance  of  alternative  problem  areas.  PATs 
developed  in  this  way  often  operate  for  a  few  months,  analyze  one  process, 
file  a  mandatory  report,  and  disband.  A  PAT  is  especially  likely  to  decay  if 
management  untrained  in  Oeming  philosophy  fails  to  act  upon  the  team's 
recommendations.  Dr  Deming  says  that  management's  unwillingness  to  respond  to 
suggestions  to  improve  the  system  is  the  main  reason  for  the  failure  of 
quality  circles  in  this  country. 

Another  common  misuse  of  the  steering  committee/PAT  approach  is  that  PATS 
usually  work  problem  areas  on  a  short-term  basis.  After  a  PAT  achieves  some 
initial  results,  such  as  submitting  frequency  data  on  "loops"  in  the  flow 
chart  of  an  important  process,  a  report  is  written  and  the  PAT  disbands.  Then 
a  different  PAT  is  established  for  another  problem  area.  The  misuse  is  that 
the  problem  area  worked  by  the  disbanded  PAT  ceases  being  systematically 
improved.  This  violates  a  central  tenant  of  Dr  Deming's  philosophy: 
continuous  and  never  ending  improvement.  Another  problem  in  the  usual 
approach  to  PATs  is  that  recommended  changes  may  help  a  process  or  department 
where  a  PAT  is  assigned  but  may  have  harmful  effects  elsewhere  in  the 
organization.  (In  his  4-day  seminar,  Dr  Deming  discusses  this  general  problem 
with  a  handout  entitled  "On  Cooperation:  An  Exercise  for  the  Working 
Groups."  The  example  in  the  handout  stresses  that  "management  allots  to 
[organizational  components]  individually  the  responsibility  for  certain  goals 
of  improvement.  Each  [component]  is  then  driven  to  actions  that  will  maximize 
its  own  individual  benefit,  without  concern  for  the  other  areas."  Another 
problem  with  the  steering  committee/PAT s  approach  is  that  workers  who  are  not 
on  a  PAT  can  be  virtually  uninvolved  in  TQM.  This  violates  Dr  Deming's  point 
14,  "Put  everyone  to  work  to  accomplish  the  transformation." 

Implementing  TQM  by  the  second  approach,  "management  teaching  Dr  Deming's 
philosophy  and  employing  a  master  statistician,"  can  also  have  limitations. 
Perhaps»4he  most  serious  limitation  is  that  this  approach  does  not  provide 
sufficient  structure  and  guidance  on  exactly  how  to  implement  TQM.  Commanders 
and  managers  who  are  highly  motivated  to  implement  TQM  confide  they  are 
uncertain  and  confused  about  exactly  what  to  do  with  this  approach.  They 
report  feeling  the  need  to  seek  the  services  of  one  of  the  very  costly  TQM 
consultants  that  have  recently  sprung  up  around  Dr  Deming  and  the  beltway. 

Even  when  such  consulting  services  are  secured,  the  assistance  turns  out  to  be 
workshops  on  the  consultant's  version  of  Dr  Deming's  14  points,  deadly 
diseases,  and  obstacles  or  Statistics  101.  Usually  no  substantive  guidance  is 
provided  in  implementation  beyond  perhaps  the  use  of  the  Steering 
Committee/PAT  approach.  Another  problem  with  the  teach  philosophy/hire  a 
statistician  approach  is  the  requirement  to  hire  a  master  statistician.  Many 
DoD  organizations,  especially  in  the  test  and  evaluation  (T&E)  and  research 
and  development  (R&D)  areas,  have  on  their  staffs  some  of  the  best  trained  and 
most  experienced  statisticians  in  the  country.  It  is  claimed,  however,  that 
these  statisticians  can't  do  the  TQM  job  because  they  were  taught  statistics 
incorrectly,  namely  they  were  incorrectly  taught  that  enumerative  statistics 
can  be  applied  to  analytic  problems.  Furthermore,  it  is  claimed  that  they 
fail  to  fully  appreciate  that  variations  in  any  process  can  be  partitioned, 
usually  with  control  charts,  into  special  (or  assignable)  and  common  causes  as 
a  basis  for  management  action  to  make  processes  predictable.  There  is  no 
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disputing  the  great  value  of  control  charts,  especially  in  assembly  line-type 
operations,  or  of  Or  Oeming's  teaching  on  special  and  common  causes.  However, 
a  past  president  of  the  prestigious  Japanese  Union  of  Scientists  and  Engineers 
(JUSE)  made  an  observation  worth  considering.  He  said  that  Japanese 
experience  shows  that  the  first  90%  of  quality  and  performance  problems  can  be 
identified  for  solution  with  6  of  the  so  called  "7  tools,"  another  5%  can  be 
identified  for  solution  with  the  other  one  of  the  7  tools,  the  control  chart, 
and  the  last  5%  can  be  identified  for  solution  with  the  so-called  "7  new 
tools,"  also  known  as  the  Taguchi  methods.  (The  list  of  7  tools  varies  but 
includes  Ishikawa  diagrams  (also  known  as  cause-and  effect  and  fishbone 
diagrams),  Pareto  charts,  checksheets,  histograms,  scatter  diagrams,  control 
charts,  stratification,  flow  charts,  run  charts,  and  various  graphs  (see 
Ishikawa  (1982)  and  Bossard  (1988)).  The  7  new  tools  include  the  relations 
diagram  method,  matrix  data  analysis  method,  PDPC  (process  decision  program 
chart)  method,  and  arrow  diagram  method  (see  Mizuno  (1988)).  Six  of  the  7 
tools  (not  including  the  control  chart)  listed  above  are  understood  by  every 
statistician,  regardless  of  training,  and  by  almost  every  person  who  sees  them 
daily  in  the  newspaper.  Our  experience  confirms  that  much  can  be  accomplished 
by  using  the  6  simple  tools.  These  tools  can  be  used  by  virtually  any 
practicing  statistician,  any  statician,  in-house  or  hired  on  a  part-time  basis 
from  the  teaching  staff  of  a  local  college.  Usin?  control  charts  and  guiding 
action  based  on  an  understanding  of  special  and  common  causes  are  very 
important,  but  experience  shows  that  enormous  improvement  can  be  achieved  by 
conscientious  people  working  with  the  6  simplier  statistical  tools  and  a  clear 
understanding  of  how  to  use  them  to  problem  detect  problems. 

Another  approach  to  implementing  TQM  is  becoming  increasingly  popular.  It 
was  developed  by  AFHRl  and  is  recommended  for  use  in  the  functional  areas  by 
Air  Force  Regulation  (Chapter  6)  25-5  and  used  in  a  number  of  Air  Force  and 
Navy  organizations.  The  approach  is  called  the  Methodology  for  Generating 
Efficiency  and  Effectiveness  Measures,  or  MGEEM.  Implementing  TQM  with  MGEEM 
not  only  avoids  the  problems  associated  with  the  more  common  approaches,  but 
provides  a  framework  that  greatly  facilitates  the  adoption  of  Dr  Deming's 
philosophy  and  teachings.  MGEEM  also  provides  commanders  and  managers  with  a 
comprehensive  performance  measurement  and  information  system  that  includes  a 
number  of  powerful  tools  for  improved  management  and  increased  worker  morale. 
An  explanations  of  how  MGEEM  facilitates  TQM  follows.  However,  the 
explanation  will  be  more  meaningful  to  readers  who  are  familiar  with  these 
materials  and  with  Dr  Deming's  work.  It  is  not  meant  to  be  a  complete 
technical  description  of  the  implementation  and  use  of  MGEEM.  This 
information  is  available  in  Tuttle  and  Weaver  (1986a,  b),  Tuttle  and  Weaver 
(1986a,  b).  Weaver  and  Looper  (1988a),  and  several  unpublished  manuscripts. 
Weaver  and  Looper,  no  date),  and  Weaver  (no  date),  all  available  from  AFHRL. 

In  the  following  explanation  of  how  MGEEM  implements  TQM,  references  to  Dr 
Deming's  14  points  (Deming, 1988,  Chapter  2)  are  in  parentheses. 

MGEEM  applies  to  organizational  components  at  every  hierarchical  level, 
including  the  commander  and  staff,  each  division,  each  branch,  and  each 
function  or  work  center.  (Point  1.  Put  everyone  to  work  to  accomplish  the 
tranformation. )  An  in-house  facilitator  leads  members  of  each  component 
through  a  series  on  consensus-seeking  excercises.  (Point  8.  Encourage 
two-way  communications  to  drive  out  fear.)  The  MGEEM  is  tailored  to  each 
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component  based  on  inputs  from  members.  First,  component  members  establish  or 
review  the  component's  mission  statement  and  may  become  involved  in  strategic 
planning.  Reviewing  the  mission  is  important  because  Dr  Deming  observes  that 
probably  80%  of  America's  workers  don't  know  their  job  and  are  afraid  to  ask 
(Mann,  1085,  p.  101).  (Point  1.  Create  Constancy  of  Purpose.)  Second, 
component  members  use  input-output  analysis  to  (a)  identify  internal  and/or 
external  custormers,  identify  customer  needs,  and  consider  that  quality  is 
defined  in  terms  of  anticipating  and  satisfying  customer  requirements  (Point 
1.  Create  constancy  of  purpose)  and  (b)  identify  suppliers  and  consider  that 

the  contribution  of  suppliers  to  quality  goes  beyond  lowest  price  to  include 

cooperation  (Point  4.  End  the  practice  of  awarding  business  solely  on  the 
basis  of  proce).  Third,  component  members  break  their  mission  statement  into 
measurable  parts,  called  key  result  areas  (KRAs),  and  develop  indicators  which 
measure  how  well  KRAs  are  being  accomplished.  Fourth,  component  members 
develop  graphics,  called  mission  effectiveness  (ME)  charts,  through  which  they 
express  their  judgment  about  the  relationship  between  feasible  levels  of 
performance  on  each  indicator  and  the  component's  overall  effectiveness. 
Performance  results  on  ME  charts  are  periodically  (monthly)  fed  back  to 

managers  for  improved  leadership  and  to  workers  in  feedback  sessions  to 

improve  morale.  Feedback  sessions  are  among  the  most  important  part  of 
MGEEM.  Workers  on  "feedback  teams"  identify  barriers  to  improvement  on  their 
ME  charts  (Point  8.  Encourage  two-way  communication  to  drive  out  fear)  and 
management  takes  action  where  appropriate  on  the  recommendations  of  the 
feedback  teams.  (Point  12.  Remove  barriers  that  rob  workers  of  their  right  to 
pride  of  workmanship.  Point  5.  Improve  constantly  and  forever  the  systems  of 
production  and  service.  Point  7.  Adopt  and  initiate  leadership).  Remember 
that  Dr  Juran  estimates  that  15%  of  organization  problems  can  be  corrected  by 
the  workers.  This  leaves  management  with  the  responsibility  for  improving  the 
other  85%  through  changes  in  the  system  (Mann,  1985,  p.  7).  Removing  barriers 
may  involve  training  (Point  6.  Institute  training  on  the  job)  or  may  require 
greater  coordination  with  another  organizational  components  (Point  9.  Break 
down  barriers  between  departments).  Managers  who  practice  Dr  Deming' s 
teachings  do  not  use  slogans,  exhortions,  targets  (Point  10),  work  standards 
(quotas)  (Point  11a),  or  management  by  objectives  (Point  lib)  to  bring  about 
improvements  on  ME  charts.  Instead  they  accept  their  responsibility  to 
constantly  and  forever  improve  the  systems  in  which  they  and  their 
subordinates  work  (Point  5).  Such  managers  realize  that  quality  is  build  in 
the  product  or  service  by  improving  the  processes  that  produced  it,  not 
through  inspection  (Point  3).  They  realize  that  in  an  ever-changing  world 
adaptable  and  innovative  workers  are  developed  by  education  and 
self-improvement  that  extends  beyond  training  for  the  skills  required  by  their 
present  jobs  (Point  13). 

Use  of  the  MGEEM  includes  two  versions  of  PATs.  First,  members  of  every 
organizational  component  received  monthly  feedback  on  how  they  performed  as  a 
group  on  their  ME  charts.  Members  in  every  organizational  component  are  a 
feedback  team  which  is  a  PAT.  This  means  there  are  permanent  "feedback  PATs" 
throughout  the  organization,  and  everyone  is  a  member  of  a  PAT  (Point  14.  Put 
everyone  to  work  to  accomplish  the  transformation).  As  feedback  PATs  improve 
constantly  and  forever  the  system  of  production  (Point  5)  they  recognize  that 
some  barriers  to  improvement  extend  beyond  the  boundaries  of  their  component 
and  authority.  Such  problems  are  worked  by  cross-functional  teams,  a  second 
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type  of  PAT,  a  "cross-functional  PAT".  Cross-functional  PATs  are  initiated  by 
management  when  a  problem  that  extends  across  the  boundaries  of  two  or  more 
organizational  components  is  identified.  (Point  9.  Break  down  barriers 
between  department).  The  membership  of  a  cross-functional  PAT  includes  an 
advocate  from  staff  who  represents  the  PAT  to  the  commander,  representati ves 
from  each  functional  component  involved  with  the  problem,  a  ““object  matter 
expert,  and  an  action  officer  responsible  for  day-to-day  measurement  and 
reporti ng. 

Neither  of  the  MGEEM  PATs  use  judgment  to  identify  barriers  or  processes 
that  require  their  attention.  Instead  the  slopes  on  ME  charts  provide  this 
guidance.  Poor  performance  (or  barrier)  on  a  steep  slope  should  usually  be 
worked  before  poor  performance  (or  barrier)  on  a  flat  slope.  In  addition, 
having  ME  charts  throughout  an  organization  reveals  if  changes  to  benefit  one 
component  have  inadvertently  caused  harmful  effects  elsewhere. 

Both  types  of  MGEEM  PATs  use  the  Deming  cycle  to  identify  and 
solve/eliminate  problems/barriers  to  performance  member  of  both  types  of  MGEEM 
PATs  are  trained  in  the  use  of  the  7  tools  (Ishikawa,  1982;  Brassard,  1988) 
and  Dr  Juran's  (1988)  PAT  team  techniques.  MGEEM  PATs  are  informed  that  90% 
of  problems  can  be  worked  successfully  with  simple  statistical  tools.  They 
are,  however,  taught  and  encouraged  to  use  control  charts  where  appropriate 
and  to  apply  Dr  Deming* s  perspective  of  special  and  common  causes  so  that  they 
don't  tamper  with  a  stable,  predictable  system. 

In  addition  to  being  a  framework  for  implementing  TQM,  MGEEM  provides 
commanders  and  managers  with  a  comprehensive  organizational  performance 
measurement  and  information  system  which  includes  a  number  of  powerful  tools 
for  improved  management  and  increased  worker  morale.  These  additional 
benefits  of  the  MGEEM  surpass  anything  offered  by  other  approaches  to 
implementing  TQM.  The  benefits  are  derived  from  use  of  the  ME  charts.  MGEEM 
is  an  organizational  performance  measurement  system  because  ME  charts  provide 
measures  for  managers  and  workers  which  allow  them  to  monitor  how  well  they 
are  doing  in  accomplishing  their  KRAs.  Furthermore,  measures  on  ME  charts  can 
be  rolled  up  so  that  higher  level  managers  can  monitor  the  overall  performance 
of  one  or  several  units.  The  measures  of  performance  used  in  the  MGEEM  can 
include  but  go  beyond  the  traditional  measures  available  from  accounting  and 
engineering.  It  is  these  nontraditional  measures,  such  as  timeliness  and 
customer  satisfaction,  that  Dr  Deming  says  are  so  important.  The  MGEEM  is  an 
information  system  because,  in  addition  to  their  measurement  capabilities,  ME 
charts  provide  signals  that  problems  exist  and  that  problems  are  solved. 

This,  of  course,  makes  PATs  more  effective  because  the  charts  point  to  problem 
areas  where  PATs  should  be  established.  ME  charts  also  measure  the  results  of 
the  PATs  efforts  to  correct  problems.  Furthermore,  ME  charts  provide  guidance 
on  resource  allocation.  Activities  with  charts  with  steep  slopes  (indicating 
that  the  activity  is  important  to  effectiveness)  are  usually  the  first  to 
benefit  from  new  resources  and  the  last  to  give  resources  up. 

No  approach  to  implementing  TQM  can  guarantee  the  transformation  of 
management  urged  by  Dr  Deming.  No  approach  can  guarantee  practice  of  the  14 
points,  avoidance  of  the  deadly  diseases,  and  overcoming  of  the  obstacles. 
However,  the  MGEEM  provides  a  series  of  opportunities  in  which  the  Deming 
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philosophy  can  take  root  and  be  practiced.  With  the  MGEEM  in  place,  however, 
it  is  necessary  to  institute  a  vigorous  in-house  program  of  education  in  Dr 
Deming's  philosophy  and  teaching.  There  are  a  variety  of  ways  to  conduct  this 
education,  but  according  to  Dr  Deming,  direct  and  continuous  management 
involvment  is  necessary.  Here  are  some  suggestions  about  how  to  conduct  this 
education.  Each  key  manager,  including  the  commander,  could  be  tasked  to  read 
a  variety  of  materials  {see  references)  on  one  of  the  14  points,  deadly 
diseases,  or  obstacles  and  prepare  a  3C-rr.ir.ut:;  briefing  on  it.  These 
briefings  should  be  video  taped  for  viewing  periodically  at  various  meetings 
such  as  feedback  sessions,  working  lunches,  staff  meetings,  and  commander's 
call.  Showing  of  the  tapes  should  be  followed  by  round  table  or  panel 
discussions.  Other  presentations  by  managers  on  other  quality  experts,  such 
as  Juran  (1988)  and  Peters  (1987),  can  be  similarly  arranged.  Education  on 
the  Deming  philosophy  must  be  routinely  included  in  all  organization 
activities  on  a  continuing  basis. 
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MGEEM:  New  Methods  of  Measuring  and  Enhancing  Organizational  Product'v'ty 


.  Charles  N.  Weaver 

Larry  T.  Looper 

A:r  Force  Human  Resources  LaDoratory  (AFHRL) 

The  AFHRL  has  recently  completed  a  series  of  research  projects  resulting 
'n  a  powerful  new  tool  for  measuring  and  enhancing  organ'zat’onal 
performance,  the  Methodology  for  Generating  Efficiency  and  Effectiveness 
Measures  or  MGEEM.  Througn  the  efforts  of  a  measurement  facilitator,  the 
MGEEM  can  be  developed  in  from  two  hours  (for  a  work  center)  to  two  days 
(for  the  headduarters  of  a  major  fiela  activity).  Organization  members  are 
guided  through  the  MGEEM  process  to  construct  a  comprehensive  performance 
measurement  system  unique  to  their  organization.  This  system  encourages  the 
creation  of  an  organizational  culture  focused  on  quality  by  employing  a 
variety  of  enhancements,  such  as  those  proposed  by  Deming  (1986),  Peters 
(1987),  and  Weaver  (in  progress),  which  continuously  improve  performance. 

Use  in  a  wide  variety  of  Air  Force  and  Navy  organizations  is 
demonstrating  that  the  MGEEM  can  significantly  increase  organizational 
performance,  is  highly  acceptable  to  organization  '’ember/participants 
because  the  implementation  process  develops  a  sense  of  ownership,  is  cost 
effective  *n  the  sense  that  it  creates  l4ttle  additional  paperwork  burden, 
satisfies  commanders'  need  for  a  device  which  comprehensively  measures 
organizational  performance,  makes  possible  vastly  improved  management, 
increases  morale,  and  is  easy  to  understand  and  use.  The  MGEEM  i$  a 
multistep  procedure  which  is  a  unique  combination  of  technologies  designed 
expecially  for  these  purposes. 

The  decision  to  use  the  MGEEM  requires  the  strong  commitment  and 
continual  visible  support  of  the  commander  of  the  target  organization  to  the 
measurement  and  enhancement  of  performance.  Implementation  requires  the 
effort  of  a  measurement  facilitator,  generally  external  to  the  organization, 
who  is  skilled  ;n  group  process  activities  (e.g.,  running  meetings, 
facil,-tating  group  discussions,  I’'stening,  and  bringing  about  consensus)  and 
who  understands  the  basics  of  organizational  performance  measurement.  After 
becoming  familiar  with  one  target  organization  through  ex^^ng 
documentation,  such  as  tne  tTss'Ch  statement,  organ'zation  structure  charts. 


work  center  descr; pt'ons,  and  conversations  wUh  organization  members  in  a 
s;te  visit,  the  facilitator  oegins  the  MGEEM  process  by  convening  Team  A. 
This  team  is  c  'posed  of  tne  commander  of  the  target  organization,  his/her 
immediate  sdbo  abates,  and  representative  customers.  The  facilitator  leads 
this  team  through  an  input-output  systems  analysis  so  that  everyone 
understands  tne  target  organization  ;n  terms  of  its  suppliers,  inputs, 
mission,  value  adding  act^-t'es,  outputs  (products),  and  customers. 
Customers  use  the  outputs  of  tne  target  organization  to  accomplish  their 
missions  and  may  oe  separate  organ ;zat;ons  or  other  branches,  divisions, 
etc.  of  tne  same  organization. 

The  fac;litator  continues  tne  MGEEM  process  by  asking  members  of  Team  A 
to  assume  that  they  are  commander  *or  a  day.  He  then  poses  to  Team  A  the 
question,  "What  does  the  Air  Force  <vNavy  or  whatever  is  appropriate)  pay 
this  organization  to  accomplish?"  Answers  are  clarified  and  prioritized 
using  the  nominal  group  technique  VNGT),  a  structured  process  for  banging 
about  consensus.  The  results,  called  key  result  areas  (KRAs),  are  the 
principal  intended  accompl isnments  of  the  organization.  Development  of  5  to 
7  KRAs  concludes  the  work  of  Team  A.  Next,  Team  8  is  formed  of  the 
commander's  subordinates  and  key  workers.  Members  of  Team  8  are  also  asked 
to  play  commander  for  a  day.  The  facilitator  sho«s  Team  8  each  KRA,  in 
turn,  and  asks  what  the  commander  needs  to  know  to  evaluate  how  well  the 
organization  is  doing  on  each.  Answers,  called  indicators,  are  again 
clarified  and  prioritized  using  the  NGT.  For  each  KRA  there  is  usually  one 
to  three  indicators.  Examples  of  KRAs  and  indicators  are  shown  in  Table  1. 


Table  1 


Organization 


KRA 


Indicator 


Jet  Engine  Repair 
Shop 

warehouse 

Engineering 


To  keep  eng-nes  in 
ready/servi ceao  le 
condition 

Make  del ; ver-es  ;n  a 
prompt/ t’ me' /  manner 

”0  como’eta  objects 
on  time 


Percent  of  inspections 
passed 


Minutes  ’■equired  to  deliver 
pr;ority  items 


Percent  of  suspenses  met 


~hus,  the  f’rst  stage  'n  the  MGEEM  process  results  'n  the  creation  of  a 
measurement  system  of  KRAs  and  'nd;cators  developed  Dy  organization  members 
w’th  tne  as^'stance  of  a  facilitator. 

’earn  3  continues  its  involvement  ;nto  the  second  stage  of  the  MGEEM  by 
constructing  grapnel  representations  of  the  relationship  between 
performance  on  eacn  indicator  and  overall  organizational  effectiveness. 

These  representations  of  team  consensus  are  called  mission  effectiveness 
;ME)  darts  and  display  not  only  current  levels  of  performance  but  also  snow 
wnere  the  organisation  can  improve  planning,  identify  productivity 
constraints,  and  increase  morale  and  productivity  through  feedback  and  goal 
setting.  An  ME  chart  is  developed  ‘'or  each  indicator  and  takes  into  account 
tnat  some  indicators  are  more  important  than  others.  The  facilitator  beg;ns 
ME  chart  construction  by  gu^ng  Team  3  to  a  consensus  about  the  feasible 
worst  and  best  values  possible  for  each  indicator  and  labels  the  X-axis  with 
the  name  of  the  indicator,  placing  the  worst  feasible  performance  value  on 
the  left  and  the  best  feasible  performance  value  on  the  right.  For 
instance,  the  X-axis  label  of  feasible  worst  and  bests  values  for  the 
indicators  in  Table  1  could  be:  (1)  for  inspect;:ns  passed,  from  40  to  100 
percent;  (2)  for  time  required  to  deliver  priority  items,  0  to  15  minutes, 
and  (3)  for  percent  of  suspense  dates  met,  65  to  95  percent.  The  Y-axis  is 
overall  mission  effect veness  labeled  from  -100  at  the  bottom  through  0  to 
>100  at  the  top  Jn  increments  of  10.  These  best  and  worst  values  constitute 
the  first  two  points  on  the  curve  on  an  ME  chart.  Figure  1  snows  the 
standard  form  of  an  ME  chart. 
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’he  facM'tator  then  poses  questions  to  Team  8  wi^ch  produce  a  tivrd  po’nt 

on  the  ME  chart  for  each  'nd’cator:  "What  ’S  the  level  on  th’s  ’nd’cator 

,<-ax;s)  that  has  no  impact  on  organ'zat'onal  effectiveness  (Y-axis)?”  or, 

« 

"What  ’evel  of  performance  won't  roc<  the  ooat?".  Another  way  to  put  this 
quest'on  ;s  "What  level  of  performance  on  this  indicator  won't  cause 
management  to  commit  more  resources  or  reduce  resources?"  This  answer  on 
the  X-axis  is  the  zero  point  on  the  Y  axis.  The  facilitator  continues  the 
questioning  Dy  as^ng  tne  i-npact  on  ef fecti veness  of  other  levels  on  the 
•nqiqator  until  tne  relationship  between  the  indicator  and  effectiveness  ■$ 
captured.  Curves  on  ME  charts  can  oe  1  inear  or  nonlinear  and  reflect 
consensus  of  "earn  8  concerning  tne  impact  on  mission  effectiveness  of 
different  levels  of  performance  on  indicators.  ME  charts  are  submitted  for 
review  and  approval  to  higher  management. 

where  the  organization  is  performing  on  each  indicator  is  provided 
monthly  to  managers  to  improve  planning  and  leadership  and  to  workers  4n 
"feedback  sessions"  to  encourage  discussion  about  how  to  ’improve  performance 
on  tne  key  indicators,  including  identifying  constraints  to  increased 
performance.  It  is  essential  that  management  take  seriously  suggestions 
wn;ch  result  from  feedback  sessions  by  ;mproving  the  systems  by  which  work 
is  accomplished.  Workers  produce  outputs  and  managers  work  continuously  to 
4 nprove  the  system.  This  may  require  a  new  philosophy  of  leadership  at  all 
levels.  Feedback  motivates  both  workers  and  management  to  higher  levels  of 
performance  and  ensures  that  managers  work  cont’nually  to  improve  the  work 
process. 

Figure  l  shows  two  completed  ME  charts. 


Tnese  charts  represent  the  impact  on  m^ss'on  effectiveness  (overall 
organizational  performance)  of  two  indicators.  Which  indicator  is  more 
important  to<  the  accomplishment  of  the  mission?  The  answer  is  Indicator  B 
oecause  the  slope  on  its  ME  chart  ;s  steeper.  Steep  slopes  mean  that 
changes  in  performance  nave  great  impact  on  the  mission.  Suppose  an 
organization  ;s  performing  to  the  far  left  on  the  horizontal  axes  of  both  ME 
Charts  in  Figure  2.  Where  should  the  manager  of  this  work  center  devote 
effort  and  resources  to  get  the  greatest  positive  impact  on  mission?  Again, 
oecause  of  the  steeper  slope,  the  answer  ;s  Indicator  B.  This  kind  of 
analysis  provides  many  otner  advantages,  such  as  early  warning  of  problems 
and  Knowledge  that  proolems  are  fixed. 

Since  ME  charts  transform  indicators  to  the  common  scale  of 
effectiveness  (the  Y-axis),  effectiveness  points  from  ME  charts  can  be 
summed  to  measure  a  work  center's  overall  performance,  different  work 
centers  can  be  directly  compared,  and  work  centers  can  be  aggregated  to 
measure  the  overall  performance  of  the  higher  level  organization.  ME  charts 
displayed  monthly  in  work  areas  reveal  to  workers  the  impact  on  overall 
effectiveness  of  the  current  level  of  their  efforts.  Also,  workers  know  how 
their  level  of  effort  compares  with  what  'S  expected.  Through  time,  they 
see  the  results  of  increased  or  decreased  efforts.  Beyond  feedback,  curves 
on  ME  charts  can  be  used  to  set  goals  for  achievement.  Transforming 
measures  on  indicators  to  overall  effectiveness  and  preparing  reports  as 
feedback  to  management  and  workers  can  be  accomplished  w,-th  the  target 
organization's  exisiting  data  automation  system. 

Air  Force  Regulation  25-5  requires  that  every  Air  Force  functional  area 
have  an  adequate  management  information  system.  The  MGEEM  is  incorporated 
in  the  Performance  Measures  Document  (PMD)  of  this  regulation  as  the 
recommended  measurement  component.  In  addition,  a  number  of  field 
activities  of  Navy  A ir  Systems  Command,  including  the  Naval  Plant 
Representative  0ff4ce  (NAVPRO)  at  McDonnell-Dougl as  Corporation  and  the 
Naval  Air  Test  Center,  have  oegun  MGEEM  implementations. 

AFHRL  nas  training  manuals  (such  as  the  Tuttle  it  Weaver,  1986,  and 
weaver  and  Looper,  ;n  press)  and  videos  available  for  fJeld  use  of  the 
MGEEM.  The  points  of  contact  on  the  MGEEM  at  AFHRi  are  Mr  ;_arry  loooer  at 
(512)  536-3942  and  Dr  Charles  N.  Weaver  at  (512)  536-3551. 
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